Browse Source

doc: restore 'dev doc.validate' - wrap tcllib dtplite as bin/dtplite.cmd

'dev doc.validate' invokes a bare dtplite command which the repl
unknown-handler resolves via auto_execok on PATH - no dtplite executable
existed (only the legacy misnamed bin/dtplite_run.bat), giving
"invalid command name dtplite".

- src/scriptapps/dtplite.tcl: LF-converted (polyglot payloads are embedded
  verbatim into the LF-only multishell output); fall back to the
  project-vendored tcllib (src/vendorlib_tcl9/<arch>/tcllib*) when the
  invoking tclsh lacks the dtplite package
- src/scriptapps/dtplite_wrap.toml: new scriptset config (tclsh nextshell
  on all platforms)
- bin/dtplite.cmd: generated polyglot via
  punk::mix::commandset::scriptwrap::multishell (checkfile-clean, LF-only)
- src/tests/shell/testsuites/binscripts/dtplite.test: artifact pins
  (LF-only, checkfile, byte-identical round-trip re-wrap) + execution
  usecases (usage error, validate good/bad file, validate directory tree,
  html generation) - 8/8 pass under runtests
- AGENTS.md (bin, src/scriptapps): record the dtplite scriptset, its test
  pin and the LF-only payload requirement
- punkproject.toml: 0.12.22 -> 0.12.23

Verified: 'dev doc.validate' in punk902z src validates all 64
src/doc .man files clean. The doctools pipeline remains interim -
punk::args is the intended doc source of truth.

Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com
master
Julian Noble 1 day ago
parent
commit
49ec22a928
  1. 14
      bin/AGENTS.md
  2. 1661
      bin/dtplite.cmd
  3. 2
      punkproject.toml
  4. 2
      src/scriptapps/AGENTS.md
  5. 15
      src/scriptapps/dtplite.tcl
  6. 15
      src/scriptapps/dtplite_wrap.toml
  7. 231
      src/tests/shell/testsuites/binscripts/dtplite.test

14
bin/AGENTS.md

@ -9,7 +9,8 @@ Built punk shell executables (kits with the punk boot layer), assorted build/exp
The `.cmd` scripts here (e.g `runtime.cmd`) are punk MULTISHELL polyglots GENERATED by
`punk::mix::commandset::scriptwrap::multishell` from scriptset sources under
`src/scriptapps/bin/` (the home for bin-deployed scriptsets, e.g `runtime.*` alongside
`getzig.*`): payload scripts (`<name>.ps1`, `<name>.bash`, ...) plus a `<name>_wrap.toml`
`getzig.*`) or `src/scriptapps/` itself (e.g `tclargs.*`, `dtplite.*`): payload scripts
(`<name>.ps1`, `<name>.bash`, `<name>.tcl`, ...) plus a `<name>_wrap.toml`
config, spliced into the `punk.multishell.cmd` template. The polyglot structure is
deliberately fragile (mutual shell-hiding tricks, LF-only endings, cmd.exe's 512-byte
label-scanner constraints) - a hand-edit can silently break one of the participating
@ -31,6 +32,17 @@ When asked to "fix bin/<name>.cmd":
re-wrap of the runtime scriptset reproduces `bin/runtime.cmd` byte for byte
(`src/tests/modules/punk/mix/testsuites/scriptwrap/multishell.test`
scriptwrap_runtime_cmd_roundtrip_no_drift), so drift in either direction fails tests.
`bin/dtplite.cmd` is pinned the same way
(`src/tests/shell/testsuites/binscripts/dtplite.test` dtplite_cmd_roundtrip_no_drift).
### dtplite (`dtplite.cmd`)
`bin/dtplite.cmd` wraps tcllib's dtplite doctools processor (payload
`src/scriptapps/dtplite.tcl`, tclsh nextshell on all platforms). It exists so the punk
repl's unknown-handler can resolve the bare `dtplite` command via PATH - `dev
doc.validate` (punk::mix::commandset::doc) depends on it. The payload falls back to the
project-vendored tcllib (`src/vendorlib_tcl9/<arch>/tcllib*`) when the invoking tclsh
has no dtplite package installed.
## Local Contracts

1661
bin/dtplite.cmd

File diff suppressed because it is too large Load Diff

2
punkproject.toml

@ -1,4 +1,4 @@
[project]
name = "punkshell"
version = "0.12.22"
version = "0.12.23"
license = "BSD-2-Clause"

2
src/scriptapps/AGENTS.md

@ -13,7 +13,7 @@ Standalone Tcl scripts that serve as entry points for Punk applications and util
- Scripts here are invoked directly by the user or by build tools, not loaded as packages.
- Wrapper configs (`.toml` files) define how scripts are packaged as platform executables.
- `<name>_wrap.toml` scriptsets are the SOURCE of the generated polyglot `.cmd` scripts in `<projectroot>/bin`, and bin-deployed scriptsets live in the `bin/` subfolder here (e.g `bin/runtime.ps1` + `bin/runtime.bash` + `bin/runtime_wrap.toml` -> `<projectroot>/bin/runtime.cmd` via `punk::mix::commandset::scriptwrap::multishell runtime -askme 0`, run from `src/scriptapps/bin`, alongside the `getzig.*` scriptset). Fixes to a `bin/*.cmd` polyglot are made in the scriptset sources and re-wrapped - never in the output file. Regenerate and commit the bin output together with payload changes: `src/tests/modules/punk/mix/testsuites/scriptwrap/multishell.test` pins the runtime scriptset round-trip byte-identical against `bin/runtime.cmd`.
- `<name>_wrap.toml` scriptsets are the SOURCE of the generated polyglot `.cmd` scripts in `<projectroot>/bin`, and bin-deployed scriptsets live in the `bin/` subfolder here (e.g `bin/runtime.ps1` + `bin/runtime.bash` + `bin/runtime_wrap.toml` -> `<projectroot>/bin/runtime.cmd` via `punk::mix::commandset::scriptwrap::multishell runtime -askme 0`, run from `src/scriptapps/bin`, alongside the `getzig.*` scriptset). Fixes to a `bin/*.cmd` polyglot are made in the scriptset sources and re-wrapped - never in the output file. Regenerate and commit the bin output together with payload changes: `src/tests/modules/punk/mix/testsuites/scriptwrap/multishell.test` pins the runtime scriptset round-trip byte-identical against `bin/runtime.cmd`, and `src/tests/shell/testsuites/binscripts/dtplite.test` pins the dtplite scriptset (`dtplite.tcl` + `dtplite_wrap.toml`, wrapped from this folder) against `bin/dtplite.cmd` the same way. Payload scripts embedded in polyglots must be LF-only - the wrap embeds payload bytes verbatim and the multishell output contract is LF-only.
- The `spud/` directory holds the spud build tool's app scripts.
- The `tools/` directory holds miscellaneous build and deployment utilities.

15
src/scriptapps/dtplite.tcl

@ -16,7 +16,22 @@
# Meta license BSD
# @@ Meta End
if {[catch {package require dtplite 1.0.5}]} {
# Fall back to the project-vendored tcllib (pure Tcl) when the invoking
# tclsh has no tcllib installed. Works both unwrapped (src/scriptapps/dtplite.tcl)
# and wrapped (<projectroot>/bin/dtplite.cmd) - each is one level below a folder
# containing (or sibling to) src/vendorlib_tcl9/<arch>/tcllib<ver>
set selfdir [file dirname [file dirname [file normalize [info script]/__]]]
foreach libdir [glob -nocomplain -types d -directory $selfdir\
../vendorlib_tcl9/*/tcllib*\
../src/vendorlib_tcl9/*/tcllib*] {
set libdir [file normalize $libdir]
if {$libdir ni $::auto_path} {
lappend ::auto_path $libdir
}
}
package require dtplite 1.0.5
}
# dtp lite - Lightweight DocTools Processor
# ======== = ==============================

15
src/scriptapps/dtplite_wrap.toml

@ -0,0 +1,15 @@
[application]
template="punk.multishell.cmd"
as_admin=false
scripts=[
"dtplite.tcl",
]
default_outputfile="dtplite.cmd"
default_nextshellpath="/usr/bin/env tclsh"
default_nextshelltype="tcl"
win32.nextshellpath="tclsh"
win32.nextshelltype="tcl"
win32.outputfile="dtplite.cmd"

231
src/tests/shell/testsuites/binscripts/dtplite.test

@ -0,0 +1,231 @@
package require tcltest
#Behaviour + artifact tests for bin/dtplite.cmd - the scriptwrap-multishell wrapped
#tcllib dtplite application (payload: src/scriptapps/dtplite.tcl + dtplite_wrap.toml).
#
#Context: 'dev doc.validate' (punk::mix::commandset::doc::validate) invokes a bare
#`dtplite validate <path>` which the punk repl unknown-handler resolves via auto_execok,
#so a dtplite executable must be findable on PATH. bin/dtplite.cmd provides that
#cross-platform (cmd.exe on windows dispatching to tclsh; sh/tclsh on unix).
#
#Covered:
# - artifact contract: bin/dtplite.cmd is LF-only and passes scriptwrap checkfile
# (no 512-byte label location errors)
# - source sync: re-wrapping the dtplite scriptset from src/scriptapps reproduces the
# committed bin/dtplite.cmd byte for byte (fix payloads and re-wrap - never the output)
# - execution usecases (windows via cmd.exe; unix via sh - both need a tclsh with the
# dtplite package or the project-vendored tcllib fallback):
# * no args -> usage error, nonzero exit
# * validate <good .man file> -> exit 0
# * validate <bad .man file> -> nonzero exit + doctools error on stderr
# * validate <directory tree> -> exit 0 (the 'dev doc.validate' usecase)
# * html generation to an output file
#
#NOTE for agents: bin/dtplite.cmd is GENERATED - edit src/scriptapps/dtplite.tcl /
#dtplite_wrap.toml and re-wrap with punk::mix::commandset::scriptwrap::multishell
#(see src/scriptapps/AGENTS.md).
namespace eval ::testspace {
namespace import ::tcltest::*
variable common {
set result ""
}
variable testdir [file dirname [file normalize [info script]]]
#<projectroot>/src/tests/shell/testsuites/binscripts -> src/tests is 3 levels up
variable testsroot [file normalize [file join $testdir .. .. ..]]
variable projectroot [file normalize [file join $testsroot .. ..]]
variable target [file join $projectroot bin dtplite.cmd]
testConstraint havedtplitecmd [file exists $target]
testConstraint iswindows [expr {$::tcl_platform(platform) eq "windows"}]
testConstraint isunix [expr {$::tcl_platform(platform) eq "unix"}]
testConstraint havescriptwrap [expr {![catch {
package require Thread ;#punk::fileline textinfo (used by checkfile) calls thread::id
package require punk::mix::commandset::scriptwrap
}]}]
proc readbytes {path} {
set fd [open $path r]
fconfigure $fd -translation binary
set data [read $fd]
close $fd
return $data
}
#write with explicit lf translation
proc writefile_lf {path content} {
set fd [open $path w]
fconfigure $fd -translation lf
puts $fd $content
close $fd
}
#run bin/dtplite.cmd with args; returns dict {exit <code> output <stdout+stderr>}
#on windows via cmd.exe (the way the repl unknown-handler launches it), on unix via sh
proc run_dtplite {args} {
variable target
set output ""
set code 0
if {$::tcl_platform(platform) eq "windows"} {
set comspec [expr {[info exists ::env(ComSpec)] ? $::env(ComSpec) : "cmd.exe"}]
set cmdline [list $comspec /c [file nativename $target] {*}$args]
} else {
set cmdline [list sh $target {*}$args]
}
if {[catch {exec -- {*}$cmdline << "" 2>@1} output opts]} {
set ec [dict get $opts -errorcode]
if {[lindex $ec 0] eq "CHILDSTATUS"} {
set code [lindex $ec 2]
} else {
set code -1
}
}
return [dict create exit $code output $output]
}
#--- fixtures: minimal good/bad doctools manpages + a small doc tree -------------
variable good_man {[manpage_begin sample_good n 1.0]
[copyright {2026}]
[moddesc {Sample}]
[titledesc {A minimal valid doctools manpage}]
[description]
[para] This is a minimal valid doctools document used for smoke testing dtplite.
[section Usage]
[para] Nothing to see here.
[manpage_end]}
variable bad_man {[manpage_begin sample_bad n 1.0]
[copyright {2026}]
[moddesc {Sample}]
[titledesc {An invalid doctools manpage - manpage_end missing}]
[description]
[list_begin itemized]
[item] an item, but the list is never closed and manpage_end is missing}
variable fixdir [makeDirectory dtplite_fixture]
writefile_lf $fixdir/sample_good.man $good_man
writefile_lf $fixdir/sample_bad.man $bad_man
variable treedir [makeDirectory dtplite_fixture_tree]
file mkdir $treedir/sub
writefile_lf $treedir/one.man [string map {sample_good tree_one} $good_man]
writefile_lf $treedir/sub/two.man [string map {sample_good tree_two} $good_man]
#--- artifact contract ------------------------------------------------------------
test dtplite_cmd_lf_only {the committed bin/dtplite.cmd uses LF line endings exclusively (cmd label scanning depends on it)}\
-constraints havedtplitecmd\
-setup $common -body {
variable target
set data [readbytes $target]
lappend result [expr {[string first \r $data] == -1}]
}\
-cleanup {
}\
-result [list 1]
test dtplite_cmd_checkfile_no_label_errors {the committed bin/dtplite.cmd passes scriptwrap checkfile - no 512-byte label location errors}\
-constraints {havedtplitecmd havescriptwrap}\
-setup $common -body {
variable target
set summary [punk::mix::commandset::scriptwrap::checkfile $target]
lappend result [string match "*ERROR: label location errors*" $summary]
#sanity: the analysis actually ran (labels enumerated)
lappend result [string match "*call-labels-found:*" $summary]
}\
-cleanup {
}\
-result [list 0 1]
test dtplite_cmd_roundtrip_no_drift {re-wrapping the dtplite scriptset from src/scriptapps reproduces the committed bin/dtplite.cmd byte for byte}\
-constraints {havedtplitecmd havescriptwrap}\
-setup $common -body {
variable projectroot
variable target
set scriptapps [file join $projectroot src scriptapps]
if {![file exists $scriptapps/dtplite_wrap.toml]} {
lappend result no_dtplite_sources
} else {
set outdir [makeDirectory dtplite_roundtrip]
set startdir [pwd]
cd $scriptapps
set res [punk::mix::commandset::scriptwrap::multishell dtplite -outputfolder $outdir -askme 0 -force 1]
cd $startdir
set fresh [dict get $res filename]
lappend result [expr {[readbytes $fresh] eq [readbytes $target]}]
#a failure here means either bin/dtplite.cmd was edited directly (fix
#src/scriptapps/dtplite.tcl or dtplite_wrap.toml instead and re-wrap),
#or the payload/template changed without regenerating bin/dtplite.cmd
}
}\
-cleanup {
}\
-result [list 1]
#--- execution usecases -----------------------------------------------------------
test dtplite_exec_no_args_usage_error {running dtplite.cmd without arguments exits nonzero with the dtplite usage message}\
-constraints havedtplitecmd\
-setup $common -body {
set r [run_dtplite]
lappend result [expr {[dict get $r exit] != 0}]
lappend result [string match "*wrong#args*" [dict get $r output]]
}\
-cleanup {
}\
-result [list 1 1]
test dtplite_exec_validate_good {validate of a well-formed .man file exits 0}\
-constraints havedtplitecmd\
-setup $common -body {
variable fixdir
set r [run_dtplite validate [file nativename $fixdir/sample_good.man]]
lappend result [dict get $r exit]
}\
-cleanup {
}\
-result [list 0]
test dtplite_exec_validate_bad {validate of a malformed .man file exits nonzero and reports a doctools format error}\
-constraints havedtplitecmd\
-setup $common -body {
variable fixdir
set r [run_dtplite validate [file nativename $fixdir/sample_bad.man]]
lappend result [expr {[dict get $r exit] != 0}]
lappend result [string match "*FmtError*" [dict get $r output]]
}\
-cleanup {
}\
-result [list 1 1]
test dtplite_exec_validate_tree {validate of a directory tree of .man files exits 0 - the 'dev doc.validate' usecase}\
-constraints havedtplitecmd\
-setup $common -body {
variable treedir
set r [run_dtplite validate [file nativename $treedir]]
lappend result [dict get $r exit]
}\
-cleanup {
}\
-result [list 0]
test dtplite_exec_html_generation {html generation from a .man file produces an html output file mentioning the manpage name}\
-constraints havedtplitecmd\
-setup $common -body {
variable fixdir
set outfile $fixdir/sample_good.html
set r [run_dtplite -o [file nativename $outfile] html [file nativename $fixdir/sample_good.man]]
lappend result [dict get $r exit]
lappend result [file exists $outfile]
if {[file exists $outfile]} {
set html [readbytes $outfile]
lappend result [string match "*sample_good*" $html]
} else {
lappend result no_output_file
}
}\
-cleanup {
file delete -force $fixdir/sample_good.html
}\
-result [list 0 1 1]
}
tcltest::cleanupTests ;#needed to produce test summary line.
Loading…
Cancel
Save