24 KiB

Raw Blame History

src/modules — Main Editable Module Source

Purpose

Source of truth for all editable Punk project modules. This is where agents should make generic module source edits. Each .tm file is a Tcl module belonging to a named package.

Ownership

Agents should edit files here as the primary development area for generic modules.
The top-level modules (non-punk namespace) include shellfilter, shellrun, overtype, textblock, patternpunk, and related packages.
The punk/ subdirectory holds all punk::* namespace modules.
Tcl-major-specific sibling trees under src/modules_tcl8/ and src/modules_tcl9/ mirror these module conventions where applicable, but their local AGENTS.md files control source selection for version-specific behavior.

Local Contracts

Module filenames use the literal suffix -999999.0a1.0.tm.
Corresponding <modulename>-buildversion.txt files hold the real version number.
The exception is punk::libunknown, which is manually versioned: its real major.minor.patch version lives in the filename (libunknown-<version>.tm) and there is no buildversion.txt. The same bump rules apply as for buildversion-tracked modules (see "Versioning And Releases"); the mechanics differ — see the manual-versioning bullet there.
#modpod-* directories contain internal files packed into .tm archives during build; do not flatten or edit them without understanding the modpod format.
_build/ directory holds build intermediates and should not be manually edited.
Always declare dependencies explicitly using package require <name> near file tops.
Prefer fully qualified namespaces when referencing external packages, such as package require tcl::zlib or package require TclOO.
Organize custom modules as namespaces mirroring directory structure, such as namespace eval punk::<modulename> or deeper paths like punk::lib::util::<somename>.
Use semantic versions that package vcompare can interpret; strip leading zeros according to tcl package version rules ie in each segment including either side of a or b.
For optional features, probe with if {[catch {package require foo}]} { ... } and degrade gracefully.
Vendormodules, bootsupport snapshots, VFS modules, and built output modules do not use the magic version scheme unless their own local docs explicitly say otherwise.
tsv shared-array names are process-global and subshell application code may use tsv for its own purposes: infrastructure tsv arrays must be punk_-prefixed (e.g punk_console, punk_console_facts). Legacy unprefixed arrays still in use (repl, codethread_*, zzzload_pkg*) predate this rule; rename them only as a coordinated change across all referencing modules.

Module Subsystems

Terminal/display modules include punk::ansi for ANSI codes, cp437, and Sixel support; punk::console; textblock for text composition and framing; and overtype for text overlay.
Shell modules include punk::repl, punk::shellrun for run/runout/runerr/runx, and punk::nav::ns for namespace browsing.
Functional and pattern modules include pattern and patterncmd; treat them as experimental and use them only with user instruction or confirmation.
Argument handling uses punk::args for processing, validation, and documentation generation; prefer it for public APIs and user-facing utilities unless the code path is performance critical.
Performance-critical procs may parse arguments manually, but must still carry an accurate punk::args (PUNKARGS) definition for documentation, with a comment in the proc referencing the definition id; use punk::args::parse whenever speed isn't critical or manual parsing would be complex.
Networking modules include punk::imap4, punk::netbox, punk::basictelnet, and punk::net::vxlan.
Plugin and capability provider/handler support uses punk::cap.

Work Guidance

Treat src/modules/ as the source of truth for generic editable Punk modules.
Expect the literal suffix -999999.0a1.0.tm and do not shorten or normalize it.
Prefer Unix-style LF line endings for .tm source files in this tree.
If the same proc exists in both src/modules/ and src/modules_tcl<major>/, prefer src/modules/ unless version-specific behavior is relevant or only the version-specific file is active.
Use deck module.new <name> or punk::mix::commandset::module::new to scaffold new modules.
Run tclsh src/make.tcl modules to build modules, or tclsh src/make.tcl project for a full build.
When adding a new proc to a .tm file, add a PUNKARGS argdoc block immediately before it — even if the proc parses arguments manually and the PUNKARGS is documentation-only. "User-facing" includes any proc in an exported namespace, not just shell commands. Developer-facing utility procs (e.g punk::lib::*) are not exempt.

Formatting And Layout

Opening braces stay on the same line for procs.
Multiline control structures may span lines for readability if Tcl syntax allows without line-continuation backslashes, such as braced expr contents or if expressions.
Line-continuation backslashes should only be used in documentation blocks for punk::args, such as punk::args::define arguments or lappend PUNKARGS elements.
In PUNKARGS and punk::args::define text blocks, preserve the embedded documentation/parser indentation already used inside the block; do not indent added help text further just to match surrounding Tcl nesting.
Proc bodies should not use line-continuation backslashes because they make debug line-number matching harder; use Tcl expand syntax where practical. Do not write hybrid forms such as dict create {*}{} \.
For literal-only Tcl values, prefer one braced literal expanded into the command; do not add per-line empty-expansion continuation markers inside the literal.
These expand-style patterns apply to dictionaries, lists, and other commands with many arguments. Preserve argument order whenever command semantics, list order, duplicate-key behavior, or display order matters.
For mixed literal and substituted values where element order does not matter, group reorderable literal values into one expanded braced literal and put substituted values on aligned continuation lines.
When substituted values or nested commands require Tcl expand-style continuation inside one command argument list, vertically align each trailing {*}{ marker in that list.
When entries in a many-argument command need inline documentation, an empty command substitution marker such as {*}[ ... ] may be used between arguments so comments can live inside the command. Comments may be on their own lines or at the end of marker lines. Keep only whitespace and comments inside those square brackets; do not put executable Tcl there. Align trailing {*}[ markers the same way as trailing {*}{ markers.
Before closeout for .tcl or .tm edits that add or alter proc bodies, search modified files for trailing \ and confirm every remaining hit is inside allowed punk::args documentation text.
For test-specific continuation rules, use src/tests/AGENTS.md.
Keep pipelines readable by aligning % var = ... and pipecase segments when practical.
Keep inline comments concise and describe intent, not mechanics.
Document non-trivial procedures and exports with the standard argdoc/PUNKARGS template below.
For documentation-only lappend PUNKARGS metadata that must not add a package require punk::args dependency or use punk::args::define, include the top-level ::punk::args::register block shown in the module template so punk::args can discover the metadata later.

Example expand-style dictionary literal; no per-line continuation markers are needed for literal values:

dict create {*}{
    a A
    b B
    c C
}

Example expand-style dictionary command with mixed literal and substituted values where key order does not matter:

dict create             {*}{
    literal_a A
    literal_c C
}   dynamic_b $BVAL     {*}{
}   dynamic_d [get_d]   {*}{
}

Example documented dictionary command using empty command substitutions for comments:

dict create                   {*}[
    # Padding character.
]   PAD [list \x80 $pad]      {*}[
    # Start of string.
]   SOS [list \x98 $sos]      {*}[
    # Control sequence introducer.
]   CSI [list \x9b $csi]      {*}[
]

Naming Conventions

Procedures: lowercase_with_underscores for internals; camelCase is allowed for public APIs where existing patterns fit.
Variables: lowercase_with_underscores; avoid single-letter names except for loop indices.
Namespaces: mirror directory structure; nested modules should reflect filesystem hierarchy.
Private helpers: prefix with _, such as _resolve_stream; do not export them.
Constants: UPPER_CASE_WITH_UNDERSCORES declared through namespace eval { variable CONSTANT value } when practical.
Keep namespace export lists alphabetized for clarity.

Procedure Documentation Template Guide

    #- comments such as this with a leading dash are descriptions for the agent of how-to/what-to to implement
    #- whereas comments without the dash are either literal comments, or if angle-bracketed, descriptions of what to put in the comment.
    namespace eval argdoc {
        lappend PUNKARGS [list {
            @id     -id     ::full::namespace::path::procedure_name
            @cmd    -name   "full::namespace::path::procedure_name"\
                -summary\
                    "Short single line procedure description"\
                -help\
                    "Procedure summary without too much overlap with documented
                    arguments below.

                    Existing
                    - include description of return value"
            @leaders
                arg1        -type any -optional 0 -multiple 0 -help\
                    "description of mandatory leading argument arg1"
            @opts
                -force      -type boolean
                -flag1      -type none -help\
                    "description of non-argument accepting flag -flag1"
                --          -type none -help\
                    "End of opts marker"
            @values -min 1 -max -1
                arg2        -type list      -optional 0 -help\
                    {Description of mandatory arg arg2
                      possibly spanning multiple lines, but each line should
                      generally be wrapped manually to keep sensibly short
                      for terminal display.
                      Help blocks may be wrapped in curly braces or double quotes.
                      See the punk::args module for details of acceptable structure.
                      }
                glob        -type string    -optional 1 -multiple 1 -help\
                    "Description of optional argument glob which can be repeated.
                      Indenting of argument -help contents should follow this example.
                      Note that we indent subsequent lines by 2 additional spaces,
                      whereas the @cmd -help does no such indenting. This provides
                      a more presentable display when calling `i procedure_name`."
        }]
    }
    proc procedure_name {args} {
        # <optional summary line to aid debug during viewing of proc body using tools like `cmdtrace` or `corp procedure_name`>
        #- agent: these comments are not literal contents of the proc - they are to be read as a guide.
        #- argument parsing implementation:
        #-  If not highly performance sensitive use punk::args to parse,
        #-    make use of `punk::args::parse $args withid ::full::namespace::path::procedurename`
        #-  namespace eval ::punk::args::register {
        #-      #use fully qualified so 8.6 doesn't find existing var in global namespace
        #-      lappend ::punk::args::register::NAMESPACES ::punk::ansi ::punk::ansi::ansistring ::punk::ansi::colour ::punk::ansi::argdoc
        #-  }
        #-  If performance sensitive or not part of a broad-concept of 'API' for the module and useful utilities,
        #-    use fast `switch` based code to parse the arguments.

        #- procedure functionality implementation
    }

Update the template with concrete details whenever functions are user-facing or complex.

Argument Order In PUNKARGS

The @leaders, @opts, and @values sections are not interchangeable — they determine the position of arguments in the synopsis and the expected parsing order:

@leaders — arguments that appear before opts (e.g interp in interp_sync_package_paths interp -libunknown 1). Use these when the proc signature has a fixed leading argument before optional flags, even if the proc parses args manually with {interp args}.
@opts — flags that appear after leaders (and before values).
@values — trailing arguments that appear after opts.

Putting a leading argument in @values instead of @leaders produces an incorrect synopsis (e.g [-libunknown <bool>] interp instead of interp [-libunknown <bool>]). Match the PUNKARGS section to the actual call site order, not just to which arguments are "required" vs "optional".

Error Handling And Logging

Prefer try { ... } on error {result options} { ... } for structured Tcl 8.6+ error handling.
Fallback pattern:

if {[catch {some_command} result]} {
    puts stderr "Error: $result"
    return -code error $result
}

For Punk pipelines, wrap risky commands inside pipecase blocks and emit descriptive messages via puts stderr or Punk logging helpers.
Never swallow errors silently; propagate with context so shell users see actionable details.

Pipeline And Functional Style Notes

Punk Pipeline functionality is experimental and should usually be avoided in agent-generated code unless the user specifically requests it.
Pipeline syntax includes forms such as var.= command1 |> command2 |> command3.
Pattern-match assignment syntax includes forms such as x@0,y@1,z@2.= list a b c, where @ targets an index and / is range-safe.
Destructuring examples include x@end/@@k1.= list {k1 aaa} {k1 bb}.
.= means following args form a command; = accepts a single value.
Use % var = ... bindings to capture intermediate values; keep names meaningful.
pipecase should list specific patterns before catch-alls to avoid hidden matches.
fun name pattern { ... } definitions should remain side-effect light; treat them as pure functions unless otherwise documented.
Keep pipelines short and composable; extract helper procs or fun definitions when they exceed about 10 logical steps.

Module Structure Expectations

# <Module description>
package require <dependencies>
#- comments such as this with a leading dash are descriptions for the agent of how-to/what-to to implement
#- whereas comments without the dash are either literal comments, or if angle-bracketed, descriptions of what to put in the comment.

namespace eval <module_namespace> {
    variable version <semver>
    namespace export public_proc1 public_proc2

    namespace eval argdoc {
        lappend PUNKARGS [list {
            @id     -id     <module_namespace>::public_proc1
            ... elided ...
            #- punk::args textblock similar to example given in Procedure Documentation Template
            #- This should exist immediately prior to each proc (or at least very close),
            #- even if it is only being used for documentation and not for parsing.
            #- If the public_proc1 proc does not use the PUNKARGS definition for parsing, then the parsing within the proc
            #- and the punk::args definitions of proc args and behaviour must be kept synchronized.
        }]
    }
    proc public_proc1 {args} {
        #- argument parsing code, using `punk::args::parse` if part of API and not performance critical
        #- if argument parsing is done in code here, the behaviour must match the arguments as defined in the PUNKARGS list created above.

        #- Implementation
    }

    namespace eval argdoc {
        lappend PUNKARGS [list {
            @id     -id     <module_namespace>::some_other_public_proc
            ... elided ...
        }]
    }
    proc some_other_public_proc {args} {
        ... elided ...
    }

    namespace eval private {
        # <basic description of namespace - optional but recommended>
        #- use of same PUNKARGS mechanism within a further nested argdoc namespace is recommended for clarity
        #- even within private namespaces.
        #- Generally privately namespaced procs will use the PUNKARGS for documentation only,
        #- unless there is no concern about the usually slight overhead of punk::args::parse.

        proc private_helper {arg1 arg2} {
            #- Private implementation
        }

        proc private_helper2 {args} {
            #- argument parsing code, using `switch`, `regexp`, etc if needed.

            #- Private implementation
        }
    }

    namespace eval utils {
        # <basic description of namespace - optional but recommended>
        #- other appropriately named sub namespaces such as 'utils' or 'lib' can be used to structure the codebase.
        #- If these form part of what can reasonably be considered the API of the module, they should
        #- be documented using the punk::args mechanism described above.

        #- more procs
    }
}

namespace eval ::punk::args::register {
    #use fully qualified so 8.6 doesn't find existing var in global namespace
    lappend ::punk::args::register::NAMESPACES <module_namespace> <module_namespace>::utils
}

# Required when PUNKARGS metadata is inert documentation and the module does not
# directly require punk::args or call punk::args::define. Register the module/API
# namespaces that punk::args should scan for PUNKARGS or PUNKARGS_aliases data.

##  Ready
package provide <module_namespace> [tcl::namespace::eval <module_namespace> {
    variable version
    #- this version number, exactly 999999.0a1.0, is a literal used in src module folders
    #- we refer to this sometimes as the magic version number
    set version 999999.0a1.0
}]
return

Type And Data Handling

Tcl is dynamically typed; emulate structural typing through argument validation at proc boundaries.
Validate user inputs with punk::args::parse, switch -exact, regexp, or Punk pipeline predicates before mutation.
Use dictionaries for structured data, or parallel lists if it is likely to be a performance win.
Tcl arrays can be used, but limit them to simple flat structures or cases where they are specifically warranted.
When bridging to binary data, such as ANSI/xbin parsing, document expected encodings and conversions.

Versioning And Releases

Stick to semantic versioning: major.minor.patch.
Use the magic version 999999.0a1.0 for source modules and add or update the actual version number in the associated <modulename>-buildversion.txt file.
Bump <modulename>-buildversion.txt whenever the module's API or behavior changes:
- Patch (e.g 0.1.0 → 0.1.1): bug fixes, internal refactors with no API change, or documentation-only updates to PUNKARGS.
- Minor (e.g 0.1.0 → 0.2.0): new flags, procs, or backward-compatible API additions.
- Major (e.g 1.0.0 → 2.0.0): breaking API changes (removed flags, changed return shapes, renamed procs without aliases).

When bumping, append changelog comment lines below the version documenting each change, prefixed with the new version number. e.g:

0.2.0
#First line must be a semantic version number
#all other lines are ignored.
#0.2.0 - merged -exclude-dirsegments + -antiglob_paths into unified -exclude-paths (backward-compat aliases retained)
#0.1.2 - fixed path_relative delegation edge case on Windows

This keeps the changelog discoverable alongside the version number without requiring a separate CHANGES file.

Bootstrap-tracked files only — punkcheck-buildversion.txt, punk/repo-buildversion.txt, punk/mix-buildversion.txt (the top-level punk::mix file only; punk::mix::util, punk::mix::cli, and punk/mix/commandset/* version independently and are not covered by this check), and punk/tdl-buildversion.txt: bump at least minor whenever a call site inside that file is updated to use a new API, even if the file's own interface is unchanged. src/make.tcl reads only these four files to classify bootsupport staleness (major=abort, minor=prompt, patch=silent-proceed); under-bumping one of them to patch when the call-site change is more significant causes make.tcl to mis-classify and either wrongly proceed without prompting or wrongly abort. See src/bootsupport/AGENTS.md "Bootsupport Staleness Handling" for the full contract.
All other modules (including punk::mix submodules): call-site updates follow ordinary judgement from the Patch/Minor/Major rules above — a behavior-preserving call-site change is a patch (or no bump if genuinely a no-op); reserve minor for changes that add capability to the module's own API. src/make.tcl does not read these versions, so there is no automated consequence, but keep the changelog accurate since it is the only record of the module's real semantic version.
Manually versioned modules (currently only punk::libunknown): the real major.minor.patch version is the filename suffix and the package provide block value; there is no <name>-buildversion.txt and the build does not stamp a version. The Patch/Minor/Major bump rules above apply identically — an agent changing such a module bumps by (1) renaming the file (git mv) to the new version, (2) updating the # Application <name> <version> Meta line, the doctools manpage_begin version and the provide-block set version, and (3) appending a changelog comment line to the version-history block in the module header (it substitutes for the buildversion.txt changelog). Before the first bump of such a module, verify nothing requires it by exact version or hardcoded filename (for punk::libunknown: punk_main.tcl and punk::repl glob libunknown-*.tm and pick the highest by vcompare, bootsupport's include_modules.config lists it by name only, and all requires are unversioned - verified 2026-07-11). New modules should use the magic version mechanism instead.
Modules with the magic version number must not appear in output paths such as <projectdir>/modules.
When referencing ranges, use bounded specs such as 1.2.3-2.0.0.
Convert loose versions to bounded form in module metadata; helper utilities exist in boot modules for this purpose.

Documentation And Comments

Primary documentation for procs should use the punk::args system, preferably within an argdoc sub-namespace using lappend PUNKARGS <list of textblocks>.
"User-facing" in the Procedure Documentation Template Guide means any proc in an exported namespace or any proc a developer might call — not just shell commands. Developer-facing utility procs are not exempt.
If a public proc parses arguments manually, keep the implementation behavior synchronized with its PUNKARGS definition.
Update relevant docs or usage notes when behavior changes.
Mention environment variables or flags required to run new features.

Verification

VS Code Tcl lint diagnostics are clean for modified .tm files when available.
New procs in modified .tm files have PUNKARGS argdoc blocks (documentation-only is acceptable if the proc parses args manually).
tclsh src/make.tcl modules completes without errors for module changes.
Module tests pass: tclsh src/tests/modules/<path>/tests/all.tcl.

Child DOX Index

punk/ — Core punk namespace modules (see punk/AGENTS.md)
opunk/ — Alternative punk namespace, voo-based classes (opunk::str, opunk::console) (see opunk/AGENTS.md)
punkcheck/ — Build/check system (punkcheck + punkcheck::cli)
test/ — Installed-module test packages (see test/AGENTS.md)

24 KiB Raw Blame History