From 6ef0693432f0b43217e3d3d262d47342f00258f3 Mon Sep 17 00:00:00 2001 From: Julian Noble Date: Fri, 10 Jul 2026 17:03:51 +1000 Subject: [PATCH] GOALS: G-049 achieved; new goals G-054 tclcore string-is parity, G-055 moduledoc regen workflow, G-056 punk::args word wrapping - G-049 flipped to achieved 2026-07-10 (acceptance verified on Tcl 9.0.3 + 8.7); detail file records the shipped design, the incomplete-vs-invalid classification judgement call, and verification results - G-054 [proposed]: ::tcl::string::is class choices harvested from the running interpreter at define time + cross-version behavioural parity pins (probing showed the static 9.0 list wrongly accepts dict on 8.6 - the real Tcl 8 target - and the hand model is otherwise behaviourally accurate incl the last-word-is-str C semantics) - G-055 [proposed]: agent-driven tclcore moduledoc regeneration workflow - Tcl source tree + version as input (TEMP_REFERENCE/tcl9 interim; retrieval deferred to buildsuites toml/G-005 era), verbatim text policy with two exceptions (synopsis notation translated to punkshell syntax; line re-folding while punk::args lacks word wrapping), real-vs-model parity probe as the verification gate, provenance recording, modelability gap scan; proving set = whole ::string ensemble + selected others incl multi-form ::after; tkcore (shared-section reuse a la tk_standardoptions) and core.tcl-lang.org projects (tcludp) to follow later - G-056 [proposed]: display-time word wrapping for punk::args help content (split-only never-join, ANSI/grapheme/double-width aware, paramindent-aligned continuations) - retires the G-055 folding exception for new work when shipped Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com --- GOALS.md | 19 ++++- goals/G-049-punkargs-parse-status-model.md | 35 ++++++++- goals/G-055-tclcore-regen-workflow.md | 91 ++++++++++++++++++++++ goals/G-056-punkargs-word-wrapping.md | 57 ++++++++++++++ 4 files changed, 200 insertions(+), 2 deletions(-) create mode 100644 goals/G-055-tclcore-regen-workflow.md create mode 100644 goals/G-056-punkargs-word-wrapping.md diff --git a/GOALS.md b/GOALS.md index 1756d300..3fac3341 100644 --- a/GOALS.md +++ b/GOALS.md @@ -339,7 +339,7 @@ Detail: goals/G-048-textblock-table-punkargs.md Goal: textblock::table parses its arguments via punk::args::parse (replacing the unvalidated dict merge at L6397), with its PUNKARGS definition covering both table-wrapper-specific options (-return, -rows, -headers) and the constructor passthrough options - the latter sourced by referencing punk::args definition blocks authored inline on the textblock::class::table class methods (the pattern used for render_to_input_line and rendertest in punk::ansi::class::class_ansi: lappend PUNKARGS [list { @id ... }] immediately before the method, punk::args::parse $args withid "..." inside the method body), so the constructor itself parses via punk::args and its documented option set is the single source of truth that textblock::table's PUNKARGS references rather than a parallel hand-typed list - retiring the "more options available - argument definition is incomplete" caveat and closing the -return -choiceprefix documentation/parsing mismatch (item #8 from the punk::args -choices audit) as a side effect. Acceptance: textblock::class::table's constructor carries an inline punk::args define block (lappend PUNKARGS, @id naming the class+method) and parses its args via punk::args::parse withid, replacing the current manual switch at L466; textblock::table's PUNKARGS references the constructor's documented options (via @id reference, a shared fragment, or a documented include mechanism - the chosen mechanism recorded in the detail file) so table's definition is complete without hand-duplicating the constructor's option list; an invalid -return value (e.g. -return tab) produces a punk::args usage error instead of silently passing through; valid inputs reach textblock::class::table new with behaviour parity to the current dict merge; the -choiceprefix 0 on -return is honoured by construction (punk::args::parse enforces it); the "NOTE: more options available - argument definition is incomplete" comment is removed; existing textblock test suites pass. -### G-049 [proposed] punk::args parse-status data model with machine-parsable cmdhelp returns +### G-049 [achieved 2026-07-10] punk::args parse-status data model with machine-parsable cmdhelp returns Scope: src/modules/punk/args-999999.0a1.0.tm (arg_error, parse error dispatch, colour-scheme handling), src/modules/punk/ns-999999.0a1.0.tm (cmdhelp), src/tests/modules/punk/args/testsuites/args/usagemarking.test, src/tests/modules/punk/ns/testsuites/ns/cmdhelp.test Detail: goals/G-049-punkargs-parse-status-model.md Goal: the information behind usage-display argument marking (which supplied arguments validated, which argument failed and why, which scheme applies) exists as a documented parse-status structure produced from a parse attempt and consumed by both arg_error renderers, and punk::ns::cmdhelp can return it machine-parsably via -return dict - replacing the transient goodargs/badarg locals, the badarg gaps in non-choice validation failures, and the stateful shared colour-array scheme handling. @@ -368,3 +368,20 @@ Scope: src/modules/punk/args-999999.0a1.0.tm (spec compiler, parse, arg_error/sy Detail: goals/G-053-punkargs-multiple-ranges.md Goal: -multiple accepts a {min max} occurrence range (mirroring -choicemultiple; max -1 unbounded) alongside the legacy booleans - so a definition can declare "at most once, repeat is an error" ({0 1}) or bounded repetition ({2 4}) instead of choosing between silent last-wins (0) and unbounded collection (1) - with boolean semantics preserved exactly, including the prepend-defaults/last-wins override idiom. Acceptance: parse raises a usage-style arity error naming the argument for occurrences outside a declared range; boolean -multiple 0/1 behaviour is unchanged (full existing punk::args suite passes untouched); the -optional/range-min reconciliation rule is documented and enforced at define time; the usage table Multi column and synopsis reflect declared ranges; -multipleunique/-multipleuniqueset compose with max>1 ranges unchanged; characterization tests cover the new forms and the value-shape rule. + +### G-054 [proposed] tclcore moduledoc: runtime-harvested 'string is' class choices with cross-version behavioural parity pins +Scope: src/modules/punk/args/moduledoc/tclcore-999999.0a1.0.tm (+ tclcore-buildversion.txt), src/tests/modules/punk/args/testsuites/args/ (new parity test), TEMP_REFERENCE/tcl9 (read-only reference) +Goal: the ::tcl::string::is definition's class choices (and the generated per-class virtual docids) are harvested from the running interpreter at define time - static choicelabels applied to classes present, generic label for unrecognized classes, a version note for dict - so accept/reject parity with the running Tcl holds on 8.6 and 9.x without hand-maintained per-version lists. +Acceptance: parse/parse_status against ::tcl::string::is and its per-class virtual ids agrees with the real interpreter's error-vs-ok outcome for a pinned probe matrix (missing args, trailing flag-like str word, option/class unique-prefix acceptance and ambiguity rejection, unknown option/class, -failindex var consumption leaving no str, per-version class presence: dict, unicode) on Tcl 9.0.x and 8.6; the rendered choices show only classes the running interp accepts; the parity test derives expectations from the live interpreter (not version arithmetic) and passes under both; existing args/tclcore suites pass; tclcore buildversion bumped with changelog. + +### G-055 [proposed] Agent-driven tclcore moduledoc regeneration workflow with behavioural parity verification +Scope: goals/G-055-tclcore-regen-workflow.md (workflow doc), src/modules/punk/args/moduledoc/tclcore-999999.0a1.0.tm (+ buildversion), src/tests/modules/punk/args/testsuites/args/ (parity pins), TEMP_REFERENCE/tcl9 (read-only source input; source retrieval mechanism deferred to the buildsuites toml configs / G-005 era) +Detail: goals/G-055-tclcore-regen-workflow.md +Goal: refreshing punk::args::moduledoc::tclcore is a documented, repeatable agent-driven workflow taking a Tcl source tree + version as input - man-page/source text carried verbatim (rearranged into @cmd -help/@examples/choicelabels, not reimagined; exceptions: synopsis notation is translated into punkshell's own more generic synopsis syntax rather than copied from the source docs, and text may be re-folded/line-wrapped to keep help display-width friendly while punk::args lacks word wrapping), one adaptive package with version-conditional definitions where released behaviour differs (8.6 vs 9.x), each regenerated command verified by a real-vs-model behavioural probe before acceptance, and source-checkin provenance recorded - designed so the same workflow extends later to tkcore (which loads on 'package require tk' and relies heavily on shared documentation sections, e.g the (default)::punk::args::moduledoc::tkcore::tk_standardoptions id) and to moduledocs for other core.tcl-lang.org projects (e.g tcludp). +Acceptance: the workflow is documented in the detail file (inputs, verbatim-text fidelity policy, probe-verification gate, provenance recording, shared-section reuse guidance for the tkcore pattern); the whole ::string ensemble plus a small selection of other commands including the multi-form ::after have been regenerated or verified through the workflow against the reference Tcl 9 sources, with parity pins added and 8.6 released-behaviour differences honoured (G-054's harvest/version-conditional techniques); an initial scan across tclcore commands identifying constructs not adequately modelable in the current punk::args system is recorded in the detail file, with each gap flagged as a candidate goal rather than worked around silently; tkcore and other-project moduledocs remain out of scope beyond the workflow being demonstrably reusable for them. + +### G-056 [proposed] punk::args display-time word wrapping for help content +Scope: src/modules/punk/args-999999.0a1.0.tm (arg_error table and string renderers, helpers), src/modules/textblock-999999.0a1.0.tm (only if cell/column-level wrap is the chosen mechanism), src/tests/modules/punk/args/testsuites/args/ (new wrapping characterization + existing rendering suites) +Detail: goals/G-056-punkargs-word-wrapping.md +Goal: punk::args help display (@cmd -help, argument -help, choicelabels) word-wraps over-width lines to the effective display width at render time - ANSI/grapheme aware, splitting long logical lines but never joining existing ones, so deliberately structured content (hand-folded blocks, choice tables, art choicelabels, indented continuations) is preserved by construction - letting definitions store unfolded verbatim text (retiring the G-055 re-folding exception for new work) while usage output stays terminal-width friendly. +Acceptance: a definition whose -help/choicelabel contains a single long unfolded line renders fully within the effective display width in both the table and -return string renderers (no truncation, no overflow, wrapped continuations indented to match the field's existing paramindent alignment); wrap-point calculation is ANSI-aware (SGR sequences measure zero width; styling in effect carries across the wrap) and grapheme/double-width aware to the same standard as existing punk::ansi/textblock width handling; existing hand-folded and structured help renders byte-identical (full existing args rendering/usagemarking/deferredhelp suites pass unchanged - never-join semantics verified by characterization tests); effective width derives from the terminal when available with the current -maxwidth 80 table default as fallback, and an explicit width option overrides; the chosen mechanism (punk::args-side pre-wrap of field text vs textblock table column wrap support) and its rationale are recorded in the detail file; G-055's folding exception is marked lapsed for new work once this ships. diff --git a/goals/G-049-punkargs-parse-status-model.md b/goals/G-049-punkargs-parse-status-model.md index 94f6984f..a99e051f 100644 --- a/goals/G-049-punkargs-parse-status-model.md +++ b/goals/G-049-punkargs-parse-status-model.md @@ -1,6 +1,6 @@ # G-049 punk::args parse-status data model with machine-parsable cmdhelp returns -Status: proposed +Status: achieved 2026-07-10 Scope: src/modules/punk/args-999999.0a1.0.tm (arg_error, parse error dispatch, colour-scheme handling), src/modules/punk/ns-999999.0a1.0.tm (cmdhelp), src/tests/modules/punk/args/testsuites/args/usagemarking.test, src/tests/modules/punk/ns/testsuites/ns/cmdhelp.test Acceptance: see GOALS.md index entry (canonical). @@ -78,3 +78,36 @@ string/table/tableobject only. - The scheme discrimination trick used by the tests (info border SGR params are the only scheme-unique style; error border shares params with info title) is itself a hint that scheme styling deserves a small documented palette rather than ad-hoc per-key overrides. + +## Outcome (achieved 2026-07-10) + +Shipped as punk::args 0.5.0 + punk::ns 0.2.0 (punkshell 0.5.0): + +- Structure producer: `punk::args::parse_status` (withid/withdef, -form, -caller) - the + structure is documented in its PUNKARGS definition. Builder/classifier live in + `punk::args::private::parse_status_build` / `parse_status_classify` / + `parse_status_formname`. Overall status classifies count/missing failure classes as + `incomplete` (count classes only below the minimum) and everything else `invalid`; + a type-failed word surfacing as missingrequiredvalue therefore reads `incomplete` + with badarg set - the per-argument statuses carry the specifics (documented). +- arg_error `-parsestatus` option; both renderers consume the structure ($argstatusd + per-arg status/hasvalue/value + receivednames for the group-parsekey check), + built internally from -badarg/-parsedargs when not supplied. Colour resolution is + per-render into a proc-local CLR array; the shared arrays are never written by scheme + merges, and the scheme switch accepts the documented `nocolour` (plus `nocolor`; + dash spellings retained for compat). +- get_dict missingrequiredvalue/missingrequiredleader raise sites carry `-badarg`. +- `punk::args::parse -caller` overrides the %caller% Get_caller frame walk (also added + to the parse cache key); parse_status defaults it to the definition's @cmd -name. +- cmdhelp: `-return dict` {origin docid cmdtype args_remaining parsestatus} on all + resolution paths (empty parsestatus for undocumented commands); both advisory-parse + branches use parse_status (uncached, minimal errorstyle) with -caller = querycommand + + consumed subcommand words; failures render via arg_error -parsestatus with $nextopts + so an explicit -scheme reaches the failure render. +- Verification: all six GAP pins flipped (usagemarking.test 17, cmdhelp.test 25 incl + 4 new cmdhelp_return_dict_*, parsestatus.test 12 new) - full punk/args (162+1skip) + and punk/ns (53) trees green on Tcl 9.0.3 and 8.7; full source-tree suite green + (known exec-14.3 core baseline failure only); make.tcl modules + projectversion OK. +- The G-046 'Bad number of leading values...' rewording item remains untouched + (coordinated, not duplicated): cmdhelp's no-supplied-words suppression is preserved + and the dict return reports scheme info for that case. diff --git a/goals/G-055-tclcore-regen-workflow.md b/goals/G-055-tclcore-regen-workflow.md new file mode 100644 index 00000000..caf9ffc6 --- /dev/null +++ b/goals/G-055-tclcore-regen-workflow.md @@ -0,0 +1,91 @@ +# G-055 Agent-driven tclcore moduledoc regeneration workflow with behavioural parity verification + +Status: proposed +Scope: goals/G-055-tclcore-regen-workflow.md (workflow doc), src/modules/punk/args/moduledoc/tclcore-999999.0a1.0.tm (+ buildversion), src/tests/modules/punk/args/testsuites/args/ (parity pins), TEMP_REFERENCE/tcl9 (read-only source input; source retrieval mechanism deferred to the buildsuites toml configs / G-005 era) +Acceptance: see GOALS.md index entry (canonical). + +## Context + +punk::args::moduledoc::tclcore was produced manually from the Tcl man pages. A +behavioural audit of 'string is' (2026-07-10, driven by the G-049 parse_status +machinery) showed the hand modelling to be surprisingly accurate - arity, the +"last word is always str" C-parser semantics, option/class prefix acceptance and +ambiguity rejection all agree with the real interpreter - but exposed the failure +mode manual transcription cannot avoid: version drift in enumerated facts. The +static class list is the Tcl 9.0 set, so 'string is dict' is wrongly accepted when +running under 8.6 (dict is 8.7+/9.x). Note 8.6, not 8.7, is the real Tcl 8 target: +8.7 is unreleased and unmaintained (its 'unicode' class never shipped in any +released Tcl; 9.0 removed it). + +The audit method generalizes and is the maintainability answer: doc *authoring* +needs judgement (the man pages are not consistently machine-parsable; some text +belongs in @examples, some in choicelabels - rearrangement yes, reimagining no), +but doc *verification* is mechanical - drive the same argument shapes through the +real command (safe, deliberately-invalid invocations) and through +punk::args::parse_status against the definition, and require error-vs-ok +agreement. G-054 applies this tactically to 'string is'; this goal makes it a +repeatable workflow. + +## Approach + +Workflow inputs: a Tcl source tree (doc/*.n man pages, generic/*.c for ground +truth) plus its version/checkin id. For now the tree is TEMP_REFERENCE/tcl9 +(reference-only fossil checkout); how source trees are fetched (git/fossil, to a +settled location) is a buildsuites/G-005-era decision - the workflow only takes +"path + version" and must stay indifferent to how the tree arrived. + +Per command: +1. Read the man page section and, where behaviour is load-bearing (arity, option + windows, enum tables), the C implementation. +2. Author/refresh the punk::args definition block - verbatim text, rearranged into + @cmd -help/@examples/argument -help/choicelabels as fits the punk::args idiom. + Exception to verbatim: synopsis notation. punkshell deliberately uses its own, + more generic synopsis syntax (as rendered by punk::ns::synopsis / @form + -synopsis), so synopsis lines and usage examples embedded in the source docs + (man-page SYNOPSIS sections, wrong-#-args strings like + "class ?-strict? ?-failindex var? str") are TRANSLATED into that syntax, never + copied in Tcl's notation. Prose and semantic descriptions stay verbatim. + Second exception: line folding. punk::args help rendering currently has no word + wrapping, so source text is re-folded manually to keep help blocks + terminal-width friendly (the existing convention of short hand-wrapped lines in + -help/choicelabel blocks). Folding changes whitespace/line breaks only - words + and their order stay verbatim. If punk::args later gains word wrapping, this + exception lapses for new work. + Enumerations the runtime can answer (class lists etc.) are harvested at define + time rather than hard-coded (G-054 technique); released-version differences + (8.6 vs 9.x) handled by version-conditional definition in the single adaptive + package - no per-major package forks. +3. Verification gate: a probe matrix (valid, incomplete, invalid, prefix, + flag-lookalike shapes) run against the real command and parse_status, asserting + error-vs-ok agreement; committed as a parity test deriving expectations from + the live interpreter, not version arithmetic. +4. Record provenance: the source checkin id the text was read from, noted with the + definition (comment) and in the module changelog. + +Modelability scan: an initial pass across tclcore commands to identify constructs +the current punk::args system cannot model adequately (e.g exotic clause grammars, +context-dependent option windows). Each gap is recorded here and flagged as a +candidate goal - not silently approximated in doc blocks. + +Proving set for acceptance: the whole ::string ensemble, plus a small selection of +other commands including the multi-form ::after. + +## Notes + +- tkcore: a started moduledoc loads on 'package require tk'. It is to be treated + with the same workflow later, but relies heavily on shared documentation + sections as currently modeled by the (default)::punk::args::moduledoc::tkcore::tk_standardoptions + id - the workflow's text-placement guidance must cover shared-section reuse, so + common option text is referenced once rather than duplicated per widget. +- Other core.tcl-lang.org projects (e.g core.tcl-lang.org/tcludp) get moduledocs + following the same pattern further down the track; out of scope here beyond the + workflow being reusable. +- The 'string is' audit artifacts (probe matrices and real-vs-model comparisons on + 9.0.3 / 8.7a6 / 8.6.13) that motivated this goal are summarized in the G-049 + session work of 2026-07-10; the C ground truth for the option-window semantics is + StringIsCmd in generic/tclCmdMZ.c (objv[2..objc-2] option loop - the final word + is unconditionally str). +- Probing policy: define-time/verification probing is limited to safe, + side-effect-free invocations of pure builtins (deliberately-invalid calls + harvesting error metadata). This does not relax cmdhelp's rule of never calling + arbitrary commands to elicit usage at query time. diff --git a/goals/G-056-punkargs-word-wrapping.md b/goals/G-056-punkargs-word-wrapping.md new file mode 100644 index 00000000..fd2e3b71 --- /dev/null +++ b/goals/G-056-punkargs-word-wrapping.md @@ -0,0 +1,57 @@ +# G-056 punk::args display-time word wrapping for help content + +Status: proposed +Scope: src/modules/punk/args-999999.0a1.0.tm (arg_error table and string renderers, helpers), src/modules/textblock-999999.0a1.0.tm (only if cell/column-level wrap is the chosen mechanism), src/tests/modules/punk/args/testsuites/args/ (new wrapping characterization + existing rendering suites) +Acceptance: see GOALS.md index entry (canonical). + +## Context + +punk::args help rendering has no word wrapping: -help and choicelabel text renders +exactly as authored, so authors hand-fold lines short to stay terminal-friendly +(the convention throughout the codebase and the module documentation templates). +That convention forced a fidelity exception in the G-055 tclcore moduledoc +regeneration workflow: imported man-page prose must be manually re-folded rather +than stored verbatim. Renderer-side wrapping removes the need - definitions can +store unfolded verbatim text and the display adapts to the terminal. + +## Approach + +Core semantics: **split-only, never-join**. The renderer wraps logical lines that +exceed the effective width but never reflows or joins existing line breaks. This +preserves deliberately structured content (hand-folded paragraphs, embedded choice +tables, textblock-frame art choicelabels, indented continuation blocks) by +construction, with no per-field opt-in/opt-out annotation machinery - and makes +"existing content renders byte-identical" a directly testable invariant. + +Width handling to the same standard as the rest of the punk display stack: +ANSI-aware (SGR sequences measure zero width; styling in effect at a wrap point +carries onto the continuation), grapheme-cluster and double-width aware +(punk::ansi / punk::char facilities). Wrapped continuations indent to the field's +existing alignment (the paramindent machinery from G-046 governs where +continuation lines sit today - wrapping must land continuations at the same +column, not at column 0). + +Effective width: terminal width when detectable (punk::console), falling back to +the current table -maxwidth 80 default; an explicit width option overrides both. + +### Mechanism alternatives (decide during the work, record here) + +- **punk::args-side pre-wrap**: wrap field text in the arg_error/usage renderers + before handing cells to textblock::table (and before the -return string + renderer's line assembly). Smaller blast radius; wrap width must anticipate the + table's computed column widths - a chicken-and-egg with column autosizing that + may need a measure pass or fixed help-column budget. +- **textblock table column wrap**: a -wrap/-maxwidth-aware column feature in + textblock::class::table cells. More general (benefits every table consumer), + heavier; interacts with existing cell padding/ansibase handling and the table's + width negotiation. If chosen, textblock's buildversion and tests are in scope. + +## Notes + +- Cross-link: [[G-055]] - its "re-folding to display width" exception to verbatim + text lapses for new work once this ships (mark it there on achievement). +- The -return string renderer has its own continuation-alignment conventions + (Description-label alignment added in G-046); wrapped continuations must respect + those, not just the table path. +- Characterization first: pin current no-wrap rendering of an over-width line + (overflow behaviour) before changing it, per the usual GAP-pin workflow.