Compare commits
37 Commits
456fa80500
...
2157c01f6c
52 changed files with 3858 additions and 497 deletions
@ -1,49 +0,0 @@
|
||||
# G-039 Investigate the orphaned-shell one-core spin on a dead console |
||||
|
||||
Status: proposed |
||||
Scope: src/modules/punk/repl-999999.0a1.0.tm (console reader/event loop and EOF/error paths), src/modules/punk/console-999999.0a1.0.tm; investigation-first |
||||
Goal: the observed failure mode - an interactive punk902z left running after its hosting terminal/console went away spins roughly a full core indefinitely (observed 2026-07-08: a 37-minute orphan with a single hard-looping thread) - is reliably reproduced and root-caused, then fixed or mitigated so a shell whose console dies exits or reaches zero-CPU idle cleanly. |
||||
Acceptance: a documented procedure reproduces the spin on the current kit (e.g. launch an interactive shell in a terminal, then kill/close the hosting terminal or conhost), or the investigation records the attempts made and what evidence would reopen it; the spinning code path is identified (prime suspect: a console read/event loop treating a dead console's immediate EOF/error as retryable without backoff or termination - adjacent to the console-EOF restart path G-038 takes ownership of); after fix/mitigation, the same procedure shows the orphaned process exiting or settling at effectively zero CPU within a short grace period, with live-console interactive behaviour unchanged; the wedge-scoring hazard note (orphans polluting process-liveness checks in test harnesses) is updated to match the outcome. |
||||
|
||||
## Context |
||||
|
||||
Observation (2026-07-08, during the G-036 investigation): a punk902z process from an earlier |
||||
interactive session (started ~37 minutes prior; its hosting terminal presumed closed) was |
||||
found consuming CPU continuously - ~1550 CPU-seconds accumulating at roughly +0.5-1 |
||||
core-seconds per wall second, with exactly ONE thread in Running state (1546s of the total) |
||||
and every other thread idle in normal waits. The process had to be killed manually. It also |
||||
polluted the wedge-scoring checks of the day (process-liveness was being used as a hang |
||||
signal), which is how it was noticed. |
||||
|
||||
Not diagnosed at the time (killed to unblock the G-036 work); no dump was taken. The |
||||
suspicion: when the hosting console goes away, a console channel read/event path returns |
||||
immediately (EOF or error) and the surrounding loop retries without backoff or a |
||||
give-up-and-exit decision. Candidate sites: the repl reader loop, the EOF branch behaviour |
||||
when reopen/restart fails or loops (note tcl_interactive would be true for a real console, |
||||
enabling the `after 1 reopen_stdin` path - see the race notes in the G-038 detail file), or |
||||
punk::console query/read helpers. |
||||
|
||||
## Approach |
||||
|
||||
1. Reproduce: launch an interactive shell in a disposable terminal and kill the host |
||||
(close the tab/window; `Stop-Process` on the terminal; for classic conhost, killing |
||||
conhost.exe; also try Windows Terminal vs conhost - behaviour may differ). Watch the |
||||
orphan's CPU and thread states (`(Get-Process punk902z).Threads` sorted by |
||||
TotalProcessorTime). Try both idle-at-prompt and mid-command states at kill time. |
||||
2. If reproduced: procdump + WinDbgX scripted stack capture of the spinning thread (the |
||||
G-036 detail file documents the working tooling pipeline and pitfalls - WinDbgX process |
||||
name is DbgX.Shell, cdb is ACL-buried, `-z/-p -c -logo` scripting works). With the spin |
||||
thread's stack, map to the retry loop and decide fix: treat dead-console EOF/error as |
||||
terminal (exit per PUNK_PIPE_EOF-like policy), or add backoff + detection. |
||||
3. Coordinate with G-038: its caller-driven restart owns the console-EOF path; the dead |
||||
console case is the failure branch of the same decision point (reopen CONIN$ fails or |
||||
the console is gone entirely) and should be designed together. |
||||
|
||||
## Notes |
||||
|
||||
- Related: G-038 (console-EOF restart ownership, reopen_stdin race notes), G-036 (dump/ |
||||
debugging tooling pipeline), tcl86-console-parked-read prior art (different mechanism, |
||||
same neighbourhood). |
||||
- Harness hygiene until fixed: verify no stray punk902z processes before scoring any |
||||
liveness-based test run. |
||||
- Archived-goal references in this file: G-036 achieved 2026-07-08 (goals/archive/G-036-tcl9-udp-console-worker-wedge.md). |
||||
@ -1,87 +0,0 @@
|
||||
# G-041 punk::args multi-form matching: automated form selection for parsing and documentation |
||||
|
||||
Status: proposed |
||||
Scope: src/modules/punk/args-999999.0a1.0.tm (parse form selection, arg_error/usage form marking), src/modules/punk/ns-999999.0a1.0.tm (cmdhelp/synopsis closest-form indication), src/tests/modules/punk/args/testsuites/ |
||||
Goal: for multi-form definitions punk::args determines which form(s) an argument list matches - parse without -form attempts all permitted forms instead of effectively form 0, -form accepts the documented list-of-forms restriction, and the documentation surface indicates the match ('i after cancel <id>' presents the cancel form; 's after cancel someid' marks the closest synopsis) - with explicit single-form restriction retained for callers that require it. |
||||
Acceptance: parsing a multiform definition (after-like fixture) without -form succeeds when the args match exactly one form (the pinned GAP tests in forms.test flip to auto-selected results); an argument list matching no form (or several) produces an error naming the candidate forms rather than a form-0 type error; -form with a list of form names/indices restricts parsing to that subset (currently an 'Expected int 0-N or one of ...' error); 'i <cmd> <args...>' and synopsis output indicate the best-matching form(s) for supplied args; explicit -form <single-name-or-index> behaviour is unchanged; the full punk::args and punk::ns suites pass. |
||||
|
||||
## Context |
||||
|
||||
Multi-form definitions (@form directive) describe commands whose argument sets are |
||||
orthogonal - the motivating example is the Tcl builtin `after`, documented with several |
||||
forms (`after ms ?script...?`, `after cancel id`, `after idle script...`, ...). The |
||||
definition and display side works: form_names are recorded, `punk::ns::synopsis` renders |
||||
every form (## FORM n headers), and `punk::args::parse -form <name-or-index>` parses |
||||
against an explicitly chosen form. |
||||
|
||||
What is missing (characterized 2026-07-08 with an after-like fixture, probe results |
||||
pinned as GAP tests in src/tests/modules/punk/args/testsuites/args/forms.test): |
||||
|
||||
- `punk::args::parse` without -form (default `*`) does NOT attempt the other forms: |
||||
arguments that match only a non-zero form (e.g. `cancel after#1`) fail with form 0's |
||||
type/arity error ("Bad number of values ... Not enough remaining values", the leading |
||||
word rejected by form 0's int type). Effectively `*` parses form 0 only. |
||||
- `-form` is documented as "Restrict parsing to the set of forms listed" and typed as a |
||||
list, but a list of two form names errors with "invalid -form value ... Expected int |
||||
0-N or one of '<names>'" - only a single name or index is accepted. |
||||
- Documentation surface: `i after cancel <id>` renders help for the default form rather |
||||
than determining that the cancel form is the closest match, and `s after cancel someid` |
||||
lists all synopses without marking the matching one. |
||||
- Internal note: punk::args::parse's own definition is multiform (withid/withdef, |
||||
discriminated by literal types) but the proc hand-parses its arguments on the happy |
||||
path, so the multiform selection machinery is not exercised internally either. |
||||
|
||||
User direction (2026-07-08): there are times when restricting to a particular form is |
||||
appropriate and times when automated selection is better - both must remain expressible. |
||||
This is an advanced improvement, deferred; the immediate prerequisite work was making the |
||||
punk::args test suite comprehensive so the current behaviour is pinned before any |
||||
form-selection logic lands. |
||||
|
||||
## Approach (sketch - to be refined when activated) |
||||
|
||||
1. Form candidacy: for -form * (or a list), attempt each permitted form's parse; classify |
||||
outcomes (clean match / match-with-defaults / type-or-arity failure). Exactly one |
||||
clean match -> auto-select. None or several -> error naming candidate forms (and for |
||||
'several', possibly a deterministic preference rule - e.g. first-declared - recorded |
||||
when decided). |
||||
2. Literal/leading-word discrimination as a fast path: many multiform commands (after, |
||||
parse itself) discriminate on a leading literal - a pre-pass on literal/literalprefix |
||||
leader types can pick the form without full parse attempts. |
||||
3. -form list support: accept a subset of names/indices as documented; single value |
||||
behaviour unchanged. |
||||
4. Documentation surface: cmdhelp/arg_error accept the supplied trailing args (cmdhelp |
||||
already parses them for goodargs marking) and use the same candidacy logic to pick or |
||||
rank forms; synopsis display marks the matching form(s). |
||||
5. Error-message quality: a no-form-matches error should show per-form failure reasons |
||||
compactly rather than only form 0's. |
||||
|
||||
Downstream consumer note (2026-07-08, G-044): the interactive command-completion/hinting |
||||
goal consumes form candidacy on PARTIAL argument lists - as the user types, the hinting |
||||
display wants "which forms are still compatible with the words so far" - so the candidacy |
||||
mechanism should be exposed as an API that accepts an incomplete argument list and returns |
||||
per-form compatibility (not only an all-or-nothing full parse). Design for that consumer |
||||
when shaping step 1. |
||||
|
||||
## Alternatives considered |
||||
|
||||
- Documenting alternate forms as separate subhelp choiceinfo entries (the trace-style |
||||
pattern) - works only when a literal subcommand word exists and fragments a command's |
||||
documentation into pseudo-subcommands; rejected as the general answer. |
||||
- Requiring callers to always pass -form - status quo for parsing, but unusable for the |
||||
interactive doc surface ('i after cancel <id>') where the user supplies args, not form |
||||
names. |
||||
|
||||
## Notes |
||||
|
||||
- Probe script: scratchpad formprobe.tcl (2026-07-08); findings encoded in |
||||
src/tests/modules/punk/args/testsuites/args/forms.test (GAP-labelled tests reference |
||||
this goal and flip when it is implemented). |
||||
- Related: G-040 (choice aliasing/doc-lookup parity - same "parser and doc surface must |
||||
agree" principle); punk::args::parse -cache interaction with per-form attempts will |
||||
need review (cache key already includes selected form). |
||||
- Incidental find during the same probing: an uncommented debug `puts stderr |
||||
">>>_get_dict_can_assign_value NOT alloc_ok..."` fired on every failed clause type |
||||
assignment (punk/args ~l.6186; its happy-path twin was already commented). Fixed |
||||
2026-07-08 (punk::args 0.2.3) - noted here because form-attempt logic will exercise |
||||
that failure path heavily. |
||||
- Archived-goal references in this file: G-040 achieved 2026-07-08 (goals/archive/G-040-punkargs-choicealiases.md). |
||||
@ -1,97 +0,0 @@
|
||||
# G-045 punk::args definition authoring ergonomics: record continuation, @cmd unindented fields, constructed-definition normalization |
||||
|
||||
Status: proposed |
||||
Scope: src/modules/punk/args-999999.0a1.0.tm (record parsing in resolve, tstr interplay, arg_error @cmd rendering), src/tests/modules/punk/args/testsuites/ (rendering.test/defquoting.test as the safety net), src/modules/punk-999999.0a1.0.tm (::punk::helptopic::define_docs de-hacked as the consumer proof) |
||||
Goal: authoring punk::args definitions no longer requires backslash line-continuations or ad-hoc workarounds for multi-line records and constructed definitions - a parser-recognised record-continuation mechanism (candidate: an unquoted trailing -& token, with the detail file recording the collision analysis and the element-count disambiguation alternative), -unindentedfields honoured for @cmd fields, and constructed (string-built) definitions able to opt into the same whole-block indent normalization file-style definitions get - with the container quoting rules (braced=literal, quoted=Tcl backslash semantics, \$\{ escape) promoted from defquoting.test into the define documentation. |
||||
Acceptance: a definition using the chosen record-continuation mechanism parses identically to its backslash-continuation equivalent (existing definitions unchanged - continuation is additive), with the token's collision rules documented and an escape/rejection story for values that legitimately match it; @cmd -help/-summary honour -unindentedfields (the rendering.test GAP rendering_unindentedfields_cmd_help_GAP flips to aligned); a constructed definition can request whole-block normalization so embedded continuation indentation behaves as in file-style definitions (the rendering_constructed_def_indent_characterization expectations updated to the chosen semantics), and ::punk::helptopic::define_docs drops its manual pre-normalization to prove it; the quoting rules from defquoting.test appear in the punk::args::define -help documentation; the full punk::args suite (128 tests incl. the rendering invariants: nesting independence, relative-indent preservation) passes with GAP tests flipped, none weakened. |
||||
|
||||
## Context |
||||
|
||||
Drafted 2026-07-09 after the tests-first characterization pass (user direction: implement |
||||
tests before changes in this area). The pain points, in the user's framing: |
||||
|
||||
- The define-block syntax deliberately avoids Tcl dict syntax for human readability, which |
||||
led to backslash line-continuations and a complex interplay between record parsing in |
||||
punk::args::resolve and tstr (especially ${...} expansions keeping whitespace aligned |
||||
during multiline substitutions). |
||||
- Constructed (string-built) definitions - ::punk::helptopic::define_docs and |
||||
punk::args::ensemble_subcommands_definition are the in-tree examples - have poor |
||||
ergonomics for getting indentation right: unlike file-style definitions they receive no |
||||
whole-block indent normalization (proven: rendering.test |
||||
rendering_constructed_def_indent_characterization - continuations at exactly 4 spaces |
||||
align, other depths leak), so builders pre-normalize by hand (undent/trim hacks, |
||||
\n-relative authoring). |
||||
- A key property of normal definitions that must be preserved: they do not depend on the |
||||
indentation of the code block they appear in (proven and pinned: |
||||
rendering_nesting_independence_plain / _tstr). |
||||
- The user's experiment used an ad-hoc `--` flag followed by a braced line-spanning empty |
||||
string as a de-facto record continuation - functional but not designed-for, and |
||||
unsatisfying. |
||||
|
||||
## The -& record-continuation candidate |
||||
|
||||
The user's proposal: a special token the record parser recognises as record continuation, |
||||
e.g. an unquoted `-&` at end of line. Design considerations recorded for implementation: |
||||
|
||||
- Collision risk is low but not zero (a value or flag legitimately spelled -&). Candidate |
||||
rules: only an UNQUOTED trailing `-&` at end of a record line counts; and/or use the |
||||
element count of the assembled record to disambiguate (an odd element count where pairs |
||||
are expected implies the trailing token is a continuation marker, not a value). The |
||||
detail decision must include an escape story for a literal trailing -& value (bracing it |
||||
suffices if 'unquoted' is part of the rule). |
||||
- Detection must happen in the record parser (resolve) before/alongside line splitting - |
||||
the same place backslash-continuation assembly effectively happens via Tcl's own parse |
||||
of braced blocks today. Unlike backslash-newline, -& survives inside braced blocks of |
||||
CONSTRUCTED definitions too - which is precisely why it helps: string-built defs cannot |
||||
use backslash-newline ergonomically (it is consumed by the building code's own quoting). |
||||
|
||||
## @cmd -unindentedfields |
||||
|
||||
Accepted but ignored today (arg_error's cmd_info rendering carries an '#unindentedfields ?' |
||||
todo; pinned by rendering_unindentedfields_cmd_help_GAP: left-margin-authored @cmd help |
||||
renders its first line +4 vs continuations). Honouring it makes left-margin authoring |
||||
available for @cmd -help/-summary as it already is for argument -help and -choicelabels. |
||||
|
||||
## Constructed-definition normalization |
||||
|
||||
Options to decide at implementation (record the choice): |
||||
- an explicit define option/directive requesting whole-block undent of the supplied text |
||||
- or automatic common-prefix normalization when a definition arrives as a single text |
||||
block (risk: changing existing constructed defs' rendering - the characterization test |
||||
documents current expectations to update deliberately) |
||||
::punk::helptopic::define_docs currently pre-normalizes by hand (undent/trim + \n-relative |
||||
help strings + -unindentedfields) - it converts to the new mechanism as the proof, and the |
||||
user's uncommitted experimental hack in punk.tm is superseded. |
||||
|
||||
## Quoting documentation |
||||
|
||||
defquoting.test pinned the container rules (braced values fully literal - $, [], two-char |
||||
\n, bare backslashes; quoted values get Tcl backslash semantics - \n newline, \\ -> \ - |
||||
with $/[] still literal; \$\{...\} renders a literal ${...} in tstr-processed blocks). |
||||
Promote these rules into the punk::args::define -help documentation so documenters (e.g. |
||||
punk::imap4's {\Deleted}/{$MDNSent} choice values) have them stated, not just tested. |
||||
|
||||
## Alternatives considered |
||||
|
||||
- Keeping backslash continuations + manual pre-normalization as the documented way - |
||||
rejected: the friction is proven by two in-tree constructed-def workarounds and the |
||||
user's punk.tm experiments. |
||||
- Tcl dict syntax for definitions - rejected long ago by design (readability). |
||||
- The ad-hoc `--` + braced-gap trick as the blessed continuation - rejected: exploits |
||||
parser behaviour that was never a contract. |
||||
|
||||
## Notes |
||||
|
||||
- Safety net: src/tests/modules/punk/args/testsuites/args/rendering.test (15 tests: |
||||
nesting independence, relative-indent preservation, unindentedfields, constructed-def |
||||
characterization, multiline tstr insertions, @dynamic) and defquoting.test (container |
||||
quoting rules). GAPs to flip here: rendering_unindentedfields_cmd_help_GAP, |
||||
rendering_constructed_def_indent_characterization (expectations updated to chosen |
||||
semantics). |
||||
- Related: G-046 (deferred -help resolution and rendering fixes - same code |
||||
neighbourhood; sequencing between the two goals is free but they touch resolve |
||||
together), G-044 (completion consumes the documentation corpus these ergonomics grow). |
||||
- Session records 2026-07-09: probe scripts renderprobe*.tcl (scratchpad), the user's |
||||
experimental define_docs hack (uncommitted punk.tm working-tree diff at the time of |
||||
drafting - superseded by this goal when implemented). |
||||
- Archived-goal references in this file: G-046 achieved 2026-07-10 (goals/archive/G-046-punkargs-deferred-help-and-fixes.md). |
||||
@ -0,0 +1,67 @@
|
||||
# G-065 Declarative vendoring: toml-declared external packages with pinning, provenance and binary gating |
||||
|
||||
Status: proposed |
||||
Scope: punkproject.toml or sibling vendor manifest (schema - settled in the work), src/modules/punk/mix/ (vendor-sync command surface), src/vendorlib/ + src/vendormodules/ + src/vfs/ (materialization targets), src/make.tcl (integration) |
||||
Goal: vendoring an external third-party package into punkshell (and, via the G-027 channel, derived projects) is driven by a toml declaration - upstream source, optional version/commit pin, trim rules - materialized and updated by one command that records retrieval provenance (upstream URL, commit/tag, retrieval date, license-verification provenance per G-063) and scans for executable binaries, refusing or warning unless punkproject.toml carries an explicit binaries-allowed override; manual drop-ins (a library folder or .tm file placed by hand, in vendor dirs or vfs) remain supported and are surfaced by audit as undeclared rather than rejected; basic vendoring requires no agent/LLM tooling. |
||||
Acceptance: a documented toml manifest schema exists and declaring a package plus running the sync command materializes it under src/vendorlib or src/vendormodules (and a vfs target where declared) on a system without agents; re-running after a pin change updates the materialized copy and an unpinned declaration records its resolved version at sync time; per-package provenance (upstream URL, commit/tag or version, retrieval date) is recorded and queryable; a declared package containing executable binaries is refused (warn-only selectable as a configured mode) unless the punkproject.toml override is present, exercised by a test or documented manual verification against a scratch package; a manually dropped-in library remains loadable and is reported as undeclared by the audit/status surface, not deleted; the ecosystem-metadata compatibility survey (teapot Meta headers, relevant TIPs, wiki.tcl-lang.org / tip.tcl.tk scan) is recorded in this file with the schema decisions it informed; src/vendorlib/tcl_oauth2_library is vendored via a declaration as the proving case; derived-project applicability (how the manifest and sync travel via G-027) has a recorded design decision - implemented or deferred with rationale. |
||||
|
||||
## Context |
||||
|
||||
Vendoring today is copy-shaped: clone the upstream (e.g. TEMP_REFERENCE/tcl_oauth2_library, |
||||
2026-07-12), trim by hand, drop the result into src/vendorlib. Nothing records where the |
||||
copy came from, what version/commit it represents, or what was trimmed - the same |
||||
provenance gap G-026 closes for local-project pulls, but for external upstreams, which |
||||
G-026 deliberately does not cover. The desirable end state resembles npm/mix/zig |
||||
dependency declaration: declare the package, optionally pin it, let tooling materialize |
||||
and update it. This is not an attempt at a general Tcl package manager, but where the |
||||
Tcl ecosystem has established metadata conventions (teapot Meta headers - already reused |
||||
by G-063 - and TIP-era distribution metadata) the schema should stay compatible; toml |
||||
remains the punkshell format (G-024 direction). |
||||
|
||||
## Approach |
||||
|
||||
- Manifest location: punkproject.toml section vs sibling file (e.g. vendor.toml) decided |
||||
in the work; either way parsed by the vendored tomlish (G-014/G-024 pattern). |
||||
- Declaration fields (candidate set): name, target area (vendorlib / vendormodules / vfs |
||||
path), upstream kind + URL (git / fossil / http archive), pin (tag / commit / version), |
||||
trim/keep rules (license, readme, examples kept by default), license expectation |
||||
(cross-checked by G-063 audit), binaries-allowed (default false). |
||||
- Sync command: punk::mix dev commandset or make.tcl subcommand (decided in the work); |
||||
agent-free by design - retrieval + trim + provenance recording only. Moduledoc |
||||
generation is explicitly out of scope here (G-068). |
||||
- Mixed mode is a contract, not a transition state: hand-dropped libraries stay legal; |
||||
the audit surface (shared with G-063's enumeration) classifies each vendored entry as |
||||
declared-in-sync / declared-stale / undeclared. |
||||
- Binary scan piggybacks the G-004 policy (root AGENTS.md binaries rule): scan the |
||||
materialized payload for executable binaries (shared libs, exes, zip-based .tm |
||||
embedding executables) before accepting it. |
||||
- Ecosystem survey is a bounded research step recorded here (Notes), informing field |
||||
names and metadata mapping - not an open-ended standards effort. |
||||
|
||||
## Alternatives considered |
||||
|
||||
- Extending G-026 to cover external upstreams - rejected: G-026 is a policy about pulls |
||||
from local sibling checkouts (dirty-checkout enforcement); external retrieval, |
||||
pinning and trim rules are a different mechanism sharing only the provenance |
||||
vocabulary. |
||||
- Requiring all vendoring to go through declarations (rejecting drop-ins) - rejected: |
||||
the user explicitly wants the mixed approach; drop-ins are surfaced, not blocked. |
||||
|
||||
## Notes |
||||
|
||||
- Related: G-004 (binary policy this enforces at vendor time), G-024 (toml direction), |
||||
G-026 (local-pull provenance sibling; include_modules.config -> toml note), |
||||
G-027 (derived-project travel), G-063 (license recording hook at vendor time), |
||||
G-066 (repackaging consumes declared packages), G-067 (artifact retrieval as an |
||||
alternative upstream kind), G-068 (moduledoc status tracked against the manifest). |
||||
- Proving case: tcl_oauth2_library. A hand-trimmed copy (keeping |
||||
LICENSE/license.terms/README/examples) was placed in src/vendorlib and load-tested |
||||
with the punk91 src shell on 2026-07-12, then deliberately removed before ever being |
||||
committed (2026-07-12) so the first real vendoring of it happens end-to-end through |
||||
the declaration - retrieval, trim rules, provenance, materialization - rather than as |
||||
a retrofit over an existing copy. The git clone in TEMP_REFERENCE/tcl_oauth2_library |
||||
remains the upstream stand-in; the hand trim is the reference expectation for the |
||||
declared trim rules' output. |
||||
- The adoption scenario (an existing manual drop-in surfaced as undeclared, then |
||||
brought under a declaration) still needs testing despite the removal - recreate it |
||||
at test time with a scratch drop-in copy. |
||||
@ -0,0 +1,40 @@
|
||||
# G-066 pkgIndex.tcl-to-.tm repackaging: lib.copyasmodule expansion with embedded metadata and distribution-unit tracking |
||||
|
||||
Status: proposed |
||||
Scope: src/modules/punk/mix/commandset/loadedlib-999999.0a1.0.tm (lib.copyasmodule), src/modules/punk/mix/ (modpod/zipkit tooling as needed), src/tests/modules/punk/mix/ (converter testsuite) |
||||
Goal: third-party pkgIndex.tcl-based packages can be repackaged as single-file .tm modules (zip-based where the payload warrants it) that embed upstream documents (LICENSE, README) and a punkshell metadata datafile giving a consistent description of upstream name, version, license and distribution-unit membership - so packages split out of a multi-package upstream (tcllib-style) record that they shipped together at upstream version X and should be upgrade-checked as a unit - with the converter handling a substantially broader class of pkgIndex.tcl scripts than today's lib.copyasmodule and refusing clearly on scripts it cannot model rather than emitting a broken module. |
||||
Acceptance: the converter repackages a proving set of at least three packages - src/vendorlib/tcl_oauth2_library plus two tcllib packages, one of which has a non-trivial pkgIndex.tcl (multiple statements, computed version, or multi-file source list) - and each resulting .tm loads via package require on the primary target runtimes (Tcl 9 kit and 8.6, or a recorded limitation referencing the G-034 code-interp constraint); each repackaged module embeds the upstream LICENSE and a metadata datafile carrying upstream name, upstream version, license indication (G-063-resolvable) and distribution-unit fields; converting several packages from one upstream project in one run records a shared distribution-unit id and version queryable from the packaged artifacts (surface decided in the work); a pkgIndex.tcl construct outside the converter's modelled class produces an explicit refusal message naming the construct; converter behaviour is covered by a testsuite under src/tests/modules/punk/mix/. |
||||
|
||||
## Context |
||||
|
||||
punkshell prefers .tm modules; the wider Tcl ecosystem mostly ships pkgIndex.tcl |
||||
libraries and may continue to. Zip-based .tm modules get single-file distribution and |
||||
can carry license/readme/metadata inside the artifact - which also makes them the |
||||
natural payload for an artifact server (G-067). `dev lib.copyasmodule` already performs |
||||
a basic conversion and has worked on some tcllib modules, but pkgIndex.tcl scripts are |
||||
arbitrary Tcl and the current handling is narrow. Multi-package upstreams introduce the |
||||
unit problem: once tcllib (or similar) packages are split into individual .tm files, |
||||
nothing records that they came from one release and should be upgraded together - no |
||||
current goal touches this. |
||||
|
||||
## Approach |
||||
|
||||
- Grow lib.copyasmodule's modelled class of pkgIndex.tcl scripts incrementally |
||||
(ifneeded lines with source/load lists, simple computed versions), with an explicit |
||||
refusal path for everything else - correctness over coverage. |
||||
- Metadata datafile format: toml, schema shared with / derived from the G-065 manifest |
||||
vocabulary so vendored-in-place and repackaged artifacts describe themselves |
||||
consistently. |
||||
- Distribution-unit: unit id = upstream project identity (e.g. "tcllib"), unit version = |
||||
upstream release; recorded per artifact and aggregable ("what units are present, are |
||||
any mixed-version"). Upgrade-together enforcement is a consumer concern (G-065 sync / |
||||
G-067 retrieval) - this goal only guarantees the data exists. |
||||
- Loading on 8.6: zip-based .tm viability in the shell code interp is G-034's subject; |
||||
this goal records the limitation rather than solving mounting. |
||||
|
||||
## Notes |
||||
|
||||
- Related: G-034 (zip modpod mounting on 8.6), G-035 (mixed .tm/pkgIndex provision |
||||
characterization - what happens when both shapes of the same package are present), |
||||
G-063 (license fields in the datafile), G-065 (manifest schema sharing), G-067 |
||||
(repackaged artifacts as the publish payload). |
||||
@ -0,0 +1,37 @@
|
||||
# G-067 Module artifact channel: publish prepared .tm modules to and retrieve from configurable artifact servers |
||||
|
||||
Status: proposed |
||||
Scope: src/make.tcl or punk::mix dev commandset (publish/retrieve surface - settled in the work), user-config (consent flag + server list, G-006 pattern), src/vendormodules/ + src/vendorlib/ (retrieval targets) |
||||
Goal: prepared .tm module artifacts (typically G-066 repackaged zipkits) can be published to a punkshell official artifact server and retrieved into punkshell or derived projects on declaration, following G-006's established patterns - consent gating by default, a default official source, user-configured alternative servers as replacement or addition, version pinning encoded in the artifact addressing, checksum verification where provided - sharing rather than duplicating G-006 machinery where it exists. |
||||
Acceptance: a retrieve operation fetches a named module artifact at a pinned version from a configured server into the project's vendor area and the module loads via package require; no retrieval happens without explicit consent (config flag or interactive prompt; non-interactive use without the flag fails with an actionable message, never downloads silently); multiple servers are configurable with a documented precedence and both replacement and additional semantics available; artifact checksums are verified when the server supplies them and a mismatch aborts the install; a publish path to the official server is documented and demonstrated (authentication mechanism decided in the work); retrieval works from a derived project, or derived-project support is recorded as deferred with rationale; the relationship to G-006's downloader (shared implementation vs parallel with recorded justification) is a recorded design decision. |
||||
|
||||
## Context |
||||
|
||||
G-006 establishes consent-gated artifact download for binary build artifacts (the |
||||
zig-built set). Modules are a distinct artifact class with the same retrieval-shaped |
||||
needs, and G-027's remote-pull question already names the G-006 channel as a candidate |
||||
transport - three goals converge on one artifact-retrieval substrate. Rather than |
||||
widening G-006's tightly binary-scoped acceptance, this goal gives modules their own |
||||
contract on the same design pattern. The publish side is what makes the official server |
||||
populatable: repackage (G-066), then push, so developers on other machines can declare |
||||
and pull (G-065) instead of re-vendoring by hand. |
||||
|
||||
## Approach |
||||
|
||||
- Addressing scheme encodes name + version (and unit id/version where the artifact |
||||
belongs to a G-066 distribution unit) so pinned retrieval is a URL construction, not |
||||
a server-side search. |
||||
- Consent and server-list configuration reuse the G-006 config surface (same keys or a |
||||
documented sibling namespace) - one consent story for all remote artifact fetching. |
||||
- Retrieval integrates as an upstream kind in the G-065 manifest ("from artifact server |
||||
X" alongside "from git repo Y"), so declared vendoring and artifact retrieval are one |
||||
developer experience. |
||||
- Official-server operation (hosting, retention, signing policy) is out of scope beyond |
||||
what publish/retrieve need to interoperate with it, mirroring G-006's stance on the |
||||
binary-artifacts repo. |
||||
|
||||
## Notes |
||||
|
||||
- Related: G-006 (pattern source and candidate shared implementation), G-027 (remote |
||||
infrastructure-pull transport candidate), G-065 (manifest integration), G-066 |
||||
(artifact payloads and distribution-unit metadata). |
||||
@ -0,0 +1,41 @@
|
||||
# G-068 Agent-assisted moduledoc generation workflow for vendored third-party libraries |
||||
|
||||
Status: proposed |
||||
Scope: goals/G-068-vendored-moduledoc-workflow.md (workflow doc), src/modules/punk/args/moduledoc/ (generated companion modules), src/tests/modules/punk/args/ (probe verification where feasible) |
||||
Goal: a vendored third-party library that neither ships nor references punk::args documentation can be given a punk::args::moduledoc companion through a documented, repeatable agent-assisted workflow - the G-055 pattern generalized beyond core.tcl-lang.org projects: upstream doc text (man pages, README, doctools) carried verbatim under the G-055 fidelity policy, synopses translated into punkshell's synopsis syntax, safe probe verification applied where feasible, upstream source/version provenance recorded with the definitions - run as a separate optional step after vendoring, so basic vendoring (G-065) never requires agent availability, with each vendored package's moduledoc status (present / absent / stale against the vendored version) trackable. |
||||
Acceptance: the workflow is documented in this file (inputs, fidelity and synopsis-translation policy by reference to G-055, probe-verification gate including the criteria for declaring probing infeasible for a command, provenance recording, and how the workflow consumes a vendored payload rather than a core source tree); a moduledoc companion produced through the workflow exists for at least one vendored library - src/vendorlib/tcl_oauth2_library as the proving case - loading alongside the untouched vendored source and surfacing in the help system; the vendoring path demonstrably completes without this step (a package vendored with no moduledoc loads and audits normally); moduledoc status per vendored package is queryable from a defined surface (G-065 manifest field or the G-063/G-064 audit surface - decided in the work) and distinguishes present, absent and stale-against-vendored-version. |
||||
|
||||
## Context |
||||
|
||||
External Tcl libraries will almost never carry punk::args documentation unless future |
||||
authors adopt it; giving them punkshell-quality help means authoring a moduledoc |
||||
companion. G-055 built the workflow shape for tclcore (verbatim fidelity, synopsis |
||||
translation, probe gates, provenance) and explicitly anticipated extension to other |
||||
projects while keeping them out of scope. Doc authoring needs judgement, so this is |
||||
likely always agent-assisted - which is exactly why it must be decoupled from basic |
||||
vendoring: a developer on a machine without agents vendors now and generates (or |
||||
receives) the moduledoc later. A published artifact (G-067) can carry its moduledoc |
||||
with it, so the generation cost is paid once per ecosystem, not once per developer. |
||||
|
||||
## Approach |
||||
|
||||
- Inputs per run: the vendored payload path + recorded upstream version (from G-065 |
||||
provenance), plus whatever docs the upstream ships (man pages like oauth2.man, |
||||
README, doctools sources). |
||||
- Companion placement: a moduledoc module under src/modules/punk/args/moduledoc/ |
||||
loading on 'package require <lib>' (tkcore pattern noted in G-055), or embedded in a |
||||
G-066 repackaged artifact - placement decision recorded in the work; the vendored |
||||
source itself is never modified (src/vendorlib contract). |
||||
- Probe verification: where commands are safe/pure enough, the G-055 real-vs-model |
||||
error/ok agreement gate applies; for network-touching or stateful commands (the |
||||
oauth2 case) the workflow documents the infeasibility criteria and falls back to |
||||
doc-fidelity review only. |
||||
- Staleness: moduledoc records the upstream version it was authored against; status |
||||
compares that to the currently vendored version. |
||||
|
||||
## Notes |
||||
|
||||
- Related: G-055 (workflow pattern source; its modelability-gap scan discipline applies |
||||
here too), G-063/G-064 (audit surfaces that could report moduledoc status), G-065 |
||||
(manifest tracking, agent-free vendoring guarantee), G-066/G-067 (carrying moduledocs |
||||
inside repackaged/published artifacts). |
||||
@ -0,0 +1,50 @@
|
||||
# G-069 Dev-time lint: cross-check punk::args record splitting against tclparser where the binary is available |
||||
|
||||
Status: proposed |
||||
Scope: lint surface (location settled in the work: punk::args dev helper, dev commandset, or scriptlib/developer script), src/modules/punk/args-999999.0a1.0.tm (split_definition_records as consumed, no new dependency), src/tests/modules/punk/args/ (capability-gated suite) |
||||
Goal: a dev-time lint cross-checks the record boundaries produced by punk::args' definition record splitter (private::split_definition_records) against the tclparser C library's 'parse command' tokenization of the same (ANSI-stripped, dialect-adjusted) text, so drift between the definition dialect and real Tcl parsing rules is caught as the dialect grows (-&, @normalize, future continuation/normalization directives) - without punk::args itself gaining any dependency on the tclparser binary. |
||||
Acceptance: a documented lint command (location decided in the work) accepts definition text and/or registered definition ids and reports record-boundary divergence between split_definition_records and tclparser 'parse command'; the comparison accounts for documented dialect features rather than flagging them (at minimum: ANSI escapes stripped before parse, -& record continuation pre-joined or classified as expected divergence); the lint is capability-gated - a clean, explicit skip/notice when the tclparser package is unavailable, never an error - and `package require punk::args` continues to succeed with no tclparser present; a run across all definitions registered in a punk9win kit shell (where the tclparser binary ships) yields zero unexplained divergences, with any real divergences found either fixed or recorded in this file; exercised by a capability-gated test or a documented manual run recorded here. |
||||
|
||||
## Context |
||||
|
||||
While implementing the -& record-continuation token (G-045, archived), tclparser's |
||||
'parse' command was considered and rejected as the splitter's parsing substrate: the |
||||
definition dialect legitimately deviates from Tcl (unbalanced-bracket ANSI escapes as |
||||
data inside quoted values, line-based base-indent semantics, the -& token), and a |
||||
binary dependency in punk::args - the module that must load everywhere, including |
||||
bootsupport contexts and plain tclsh - inverts the G-004 direction. The full analysis |
||||
is recorded in goals/archive/G-045-punkargs-authoring-ergonomics.md (Progress, |
||||
increment 3) and in the punk::args 0.7.0 changelog. |
||||
|
||||
The rejected option still has value as a diagnostic: records are shaped like Tcl |
||||
commands (newline-delimited, quote/brace/backslash continuation), so where the binary |
||||
is available, parse-command boundaries are an independent oracle for the splitter. |
||||
As the dialect grows, an automated cross-check catches unintended divergence early - |
||||
the splitter's own testsuites (recordcontinuation.test, normalize.test, defquoting.test, |
||||
rendering.test) pin known behaviour but cannot flag novel drift against Tcl semantics. |
||||
|
||||
## Approach |
||||
|
||||
- Comparison pipeline: take the definition text as the splitter receives it, produce |
||||
splitter records; separately ansistrip the text, pre-join -& continuations (or |
||||
classify them), then iterate tclparser 'parse command' to produce boundary ranges; |
||||
compare record counts and boundary line positions, reporting divergences with |
||||
context. |
||||
- Corpus: registered definitions via punk::args introspection (update_definitions + |
||||
raw defs), plus ad-hoc text input for testing definition snippets during dialect |
||||
work. |
||||
- Home: this is developer tooling, not runtime behaviour - candidates are a |
||||
punk::args dev/diagnostic namespace proc (loaded lazily), a dev commandset command, |
||||
or a scriptlib/developer script following goals_lint.tcl precedent. Decide in the |
||||
work; the constraint is only that punk::args' runtime footprint is unchanged. |
||||
- The tclparser binary currently ships only in the punk9win kit (lib_tcl9); the lint |
||||
is expected to run there (or any environment with the package installed). If G-070 |
||||
(pure-Tcl tclparser) lands, the gate widens automatically if the lint requires |
||||
'parser' via ordinary package resolution. |
||||
|
||||
## Notes |
||||
|
||||
- Related: G-045 (archived - origin of the rejected-then-repurposed idea and the |
||||
dialect features the comparison must understand), G-070 (pure-Tcl tclparser would |
||||
let the lint run without the binary), G-004 (why the runtime must stay |
||||
dependency-free). |
||||
@ -0,0 +1,56 @@
|
||||
# G-070 Pure-Tcl tclparser: parse-command API fallback with behavioural parity against the C library |
||||
|
||||
Status: proposed |
||||
Scope: src/modules/punk/lib-999999.0a1.0.tm (tclparser_tcl stub + dispatch; new module if size warrants - decided in the work), src/tests/modules/punk/lib/ (parity + fallback suites), TEMP_REFERENCE/ (tclparser reference source, user-provided, read-only) |
||||
Goal: punk::lib's script-analysis machinery (tclword_to_scriptlist and the dependent analysis paths) runs on runtimes without the tclparser C binary, via a pure-Tcl implementation of the tclparser 'parse' command API covering at least the subcommands and token shapes punk::lib consumes - with behavioural parity verified against the C library rather than assumed, and the C parser still preferred where present (performance). |
||||
Acceptance: the punk::lib tclparser_tcl error stub is replaced by a working pure-Tcl implementation of the 'parse' subcommands punk::lib currently uses ('parse command' at minimum; the full covered set enumerated in this file during the work), returning the same parse-tree shapes (ranges, token types, expansion handling) the C library returns for those calls; a parity testsuite compares pure-Tcl output against the C tclparser across a recorded probe corpus (representative punkshell module scripts plus edge cases: {*} expansion, comments, backslash-newline continuation, nested command substitution, braces/quotes in words) - parity tests capability-gated on the binary, pure-Tcl-only tests running everywhere; a documented punk::lib analysis entry point (e.g. tclword_to_scriptlist) demonstrably works under a plain tclsh with no parser binary on the package path; dispatch prefers the C parser when available with the fallback engaging automatically otherwise; the reference source's location under TEMP_REFERENCE and its provenance (origin URL, version/checkin) are recorded in this file. |
||||
|
||||
## Context |
||||
|
||||
punk::lib uses the tclparser C library's 'parse command' for script analysis (word |
||||
-> scriptlist conversion, argument/expansion classification - the G-019 |
||||
dependency-scan direction). The binary ships only in the punk9win kit's lib_tcl9, |
||||
so the analysis paths are unavailable on plain tclsh, other kits and other |
||||
platforms - and lib-999999.0a1.0.tm carries an explicit placeholder: |
||||
tclparser_tcl (currently an error stub telling the user to install the C library) |
||||
plus a recorded deliberation at tclscript_to_toplevelinfo noting "we don't have a |
||||
pure-tcl implementation of 'parse' available as a fallback". A tested pure-Tcl |
||||
implementation removes the binary constraint from every current and future consumer |
||||
and aligns with the G-004 no-committed-binaries direction. It also widens where the |
||||
G-069 splitter cross-check lint can run. |
||||
|
||||
The user will place the latest obtainable tclparser source (C implementation and |
||||
any docs/tests) under TEMP_REFERENCE as the read-only behavioural reference |
||||
(candidate upstreams: the aspect fossil fork at chiselapp and the ActiveState |
||||
teapot tree, both already noted in the tclparser_tcl stub comments; punkshell also |
||||
carries a moduledoc for the parse command at |
||||
src/modules/punk/args/moduledoc/parser-999999.0a1.0.tm). |
||||
|
||||
## Approach |
||||
|
||||
- Scope the API by consumption, not by the full tclparser surface: enumerate the |
||||
subcommands and result fields punk::lib actually reads (parse command ranges, |
||||
token lists, restRange handling, simple-expansion trees), implement those first, |
||||
and record the covered/uncovered set here. Other subcommands (expr, varname, |
||||
list) follow only if a consumer needs them. |
||||
- Parity-first workflow (G-055 spirit): build the probe corpus and the |
||||
compare-against-C harness before/alongside the implementation, so every |
||||
implemented construct lands with a parity pin. Corpus draws on real punkshell |
||||
module sources (rich in expansions, multi-line commands and comments) plus |
||||
targeted edge cases. |
||||
- Dispatch: a thin front (in punk::lib or the new module) that package-requires the |
||||
C parser and falls back to the pure-Tcl engine; consumers call one entry point. |
||||
Performance note recorded rather than chased - the C library stays preferred |
||||
where installed. |
||||
- Placement: the stub lives in punk::lib today; a parser is a sizeable, |
||||
separately-testable unit, so a dedicated module (name TBD, e.g. under punk::) |
||||
with punk::lib delegating is the likely shape - decided in the work. |
||||
|
||||
## Notes |
||||
|
||||
- Related: G-019 (dependency-scan module trimming - the main analysis consumer), |
||||
G-004 (binary-free direction), G-069 (lint gains a binary-free substrate), |
||||
G-055 (parity-verification workflow precedent). |
||||
- The tclparser_tcl stub's partial char-loop sketch (in_dq/in_cb/escape state) is a |
||||
starting point only; the reference source's Tcl_ParseCommand semantics are the |
||||
contract. |
||||
@ -0,0 +1,68 @@
|
||||
# G-072 punk::args compound clause types: named alternates with per-element typing and per-alternate arity (try-class handlers) |
||||
|
||||
Status: proposed |
||||
Scope: src/modules/punk/args-999999.0a1.0.tm (type-expression parsing, clause allocation, synopsis/help renderers), src/modules/punk/args/moduledoc/tclcore-999999.0a1.0.tm (::try as proving consumer; ::if/::switch as touched), src/tests/modules/punk/args/testsuites/args/ |
||||
Goal: heterogeneous repeating clauses - try's on/trap handlers are the motivating case - can be modelled with per-alternate element types, choices and arity: 'try {} on bogus {e} {s}' is rejected with the code choice set displayed, alternates may differ in element count, mixed-order interleaving keeps parsing, and synopsis/help render each alternate's shape distinctly instead of a conflated element - with the definition mechanism (named compound types, composition operators, cross-package scoping) decided and recorded, informed among others by set-theoretic type composition. |
||||
Acceptance: a definition mechanism (decided in the work, starting from the sketches embedded in the ::try moduledoc notes: named compound types with arity, @typeimport, composition operators) lets the ::try handler be modelled so that parse rejects 'on bogus ...' displaying the on-code choices (ok/error/return/break/continue or integer) and types the trap pattern as a list, while mixed-order on/trap interleaving and finally positioning keep parsing (the 2026-07-12 probe matrix stays green); alternates of differing arity are expressible and exercised by at least one fixture (per-alternate arity derived or declared - decision recorded); synopsis and help output render each alternate's element shape distinctly (the deliberately-unpiped oncode_or_trappattern -typesynopsis workaround retired); type-name scoping across documentation packages has a recorded decision (namespacing/@typeimport mechanism, or rejection with rationale); the ::try moduledoc adopts the mechanism with real-vs-model parity probes added (::if/::switch adopt it only where it improves their models); definitions not using compound types behave unchanged (full punk::args suite passes, none weakened). |
||||
|
||||
## Context |
||||
|
||||
The ::try moduledoc models handlers as a single 4-element clause |
||||
{literal(on)|literal(trap) string list string} -multiple 1. Probing (2026-07-12, |
||||
G-041 prework) confirmed this parses everything structural correctly - mixed |
||||
on/trap order, finally positioning - but over-accepts semantically: the second |
||||
element cannot be typed per-alternate, so 'on bogus {e} {s}' passes where Tcl |
||||
errors with the valid completion codes. The moduledoc carries extensive embedded |
||||
design notes at ::try recording the difficulty and sketching mechanisms (named |
||||
compound types with arity indicators such as <on_handler:4>|<other_handler:3>, |
||||
per-definition vs global type registries, @typeimport for cross-package reuse, |
||||
RPN composition like {stringstartswith a stringendswith z AND int OR}). |
||||
Related over-acceptance findings from the same probes are listed in |
||||
goals/G-055-tclcore-regen-workflow.md Notes (reserved-word clause allocation, |
||||
'-' fallthrough position constraints) - candidates to fall out of the same |
||||
mechanism or be recorded as out of scope here. |
||||
|
||||
## Set-theoretic types input (user direction 2026-07-12) |
||||
|
||||
Investigate whether gradual set-theoretic types as adopted by Elixir |
||||
(https://elixir.hexdocs.pm/main/gradual-set-theoretic-types.html) can inform the |
||||
mechanism. Brief examination at drafting time: |
||||
|
||||
- Elixir treats types as SETS of values composed with union (or), intersection |
||||
(and) and negation (not); literal values are singleton types; dynamic() effects |
||||
gradual typing; multi-clause functions are intersections of arrow types. |
||||
- Mapping to punk::args: -type's existing | is already set union and |
||||
literal(word) is already a singleton type - the existing vocabulary is a |
||||
fragment of a set-theoretic algebra, so extension is consistent rather than a |
||||
redesign. punk::args' 'any' plays the dynamic() role. |
||||
- Negation would directly express the reserved-word gap recorded in G-055's |
||||
notes: an if else-script is "script and not literal(elseif)" - i.e. deny-lists |
||||
for clause members become type negation instead of a bespoke feature. |
||||
- Intersection covers the composite constraints the ::try notes sketched as RPN: |
||||
{stringstartswith a stringendswith z AND int OR} reads as |
||||
((startswith(a) and endswith(z)) or int) - infix/functional composition on set |
||||
semantics can replace the RPN idea. |
||||
- Named compound types then become type ALIASES over such compositions, and a |
||||
heterogeneous clause is a UNION of tuple-like clause shapes whose arity derives |
||||
from each shape's length - answering the notes' open question about declaring |
||||
arity for alternates of differing length (arity is per-alternate and computed, |
||||
not separately declared). |
||||
- A full algebra is likely overkill for the immediate try case; the design should |
||||
pick operator spellings consistent with set-theoretic composition so later |
||||
extension (negation for reserved words, intersection for composite string |
||||
constraints) slots in without breaking earlier definitions. |
||||
|
||||
## Notes |
||||
|
||||
- Related: G-041 (form selection and clause allocation interact; its candidacy |
||||
machinery should not need changes from this goal's types), G-071 (allocation |
||||
correctness for optional elements, achieved 2026-07-12 - see |
||||
goals/archive/G-071-punkargs-optional-allocation.md; its choiceword_match |
||||
allocation screen and allocation.test fixtures are the base this goal's |
||||
alternates ride on), G-055 (modelability findings list; its parity workflow |
||||
verifies whatever this goal makes expressible), G-053 (occurrence arity - |
||||
adjacent clause machinery). |
||||
- Display cost matters: the ::try notes warn bracketed alternate forms "get |
||||
unwieldy in synopsis listings" - synopsis rendering of compound types is part |
||||
of the mechanism's acceptance, not an afterthought. |
||||
- Archived-goal references in this file: G-041 achieved 2026-07-13 (goals/archive/G-041-punkargs-form-matching.md). |
||||
@ -0,0 +1,64 @@
|
||||
# G-073 punk::args unavailable choices: displayed with notes and prefix-reserving, but rejected with a tailored message |
||||
|
||||
Status: proposed |
||||
Scope: src/modules/punk/args-999999.0a1.0.tm (spec key, choiceword_match pool, choices rendering in arg_error table+string renderers, validation message), src/modules/punk/args/moduledoc/tclcore-999999.0a1.0.tm ('string is' forward-class adoption + per-class virtual id), src/tests/modules/punk/args/testsuites/args/ (new suite + tclcoreparity.test exemption) |
||||
Goal: an argument definition can declare choices that are recognised but not available in the current runtime (new key, candidate spelling -choiceunavailable <list>): they display among the choices with their ordinary -choicelabels notes (visually distinguished), participate in prefix disambiguation like -choiceprefixreservelist phantoms, and are rejected at parse with a tailored message identifying the unavailability rather than the generic listed-values error - adopted by the tclcore 'string is' model so that Tcl 8.6 shows the dict class annotated "(class not present in Tcl 8.6)", rejects 'string is dict' informatively, treats 'string is di' as ambiguous (deliberately stricter than real 8.6, preparing 8.6 users for 9.x where dict/digit/double collide), and documents 'i string is dict' via a per-class virtual id generated from the static description. |
||||
Acceptance: a definition using the new key renders unavailable entries distinguishably among the choices (mechanism decided in the work: dedicated group heading, dim/warn styling, or marker) with their -choicelabels notes, in both the table and string renderers, and synopsis choice-literal display excludes them; parse/parse_status reject an unavailable choice word - exact, or a unique prefix resolving only to it - with a message identifying it as recognised-but-unavailable rather than the generic choice error; unavailable entries participate in prefix disambiguation via choiceword_match (a prefix shared between an available and an unavailable choice is rejected as ambiguous) and are honoured identically by the punk::ns doc walk and the G-071 allocation screen (parity free via the shared resolver - no second matching rule); value-in-effect/goodarg highlighting never marks an unavailable entry; the tclcore 'string is' definition adopts the key on runtimes lacking known forward classes (dict on 8.6; the forward list is explicitly curated in the definition - unicode deliberately excluded as never-released), with a per-class virtual id generated from the static description so 'i string is dict' documents the class with its note on 8.6; the deliberate strictness divergence (real 8.6 accepts 'string is di' as digit; the model rejects it as ambiguous) is recorded in tclcoreparity.test as a user-sanctioned exemption class rather than silently special-cased, with full-word behaviour staying parity-true on both versions; definitions without the key behave unchanged (full punk::args and punk::ns suites pass). |
||||
|
||||
## Context |
||||
|
||||
User requirement 2026-07-12: forward compatibility for the 8.6-vs-9 'string is |
||||
dict' difference. The G-054 harvest (archived) makes the class choices accurate |
||||
per-runtime, so 8.6 neither displays nor mentions dict - but an 8.6 programmer |
||||
who habituates to 'string is di' (unique prefix of digit on 8.6) writes code that |
||||
breaks on 9.x where dict makes 'di' ambiguous. The requirement: 8.6 should SEE |
||||
dict (with an unavailability note - the description text with its "(class not |
||||
present in Tcl 8.6)" annotation already exists in the static |
||||
string_is_class_descriptions dict, it just never renders on 8.6), have dict |
||||
participate in prefix disambiguation (so 'di' is ambiguous on 8.6 too), and still |
||||
reject dict as an actual argument (real 8.6 rejects it). |
||||
|
||||
Capability assessment (2026-07-12/13): the display layer is ready - choicelabels |
||||
notes, choicegroups headings and choiceinfo markers all exist; the gap is purely |
||||
that choices-cell membership implies parse acceptance under -choicerestricted 1. |
||||
The trie mechanics exist as -choiceprefixreservelist (phantom prefix-calculation |
||||
members, G-040), but reservelist phantoms deliberately do not display (the |
||||
punk::help c/to/tc phantoms must stay invisible) and reject with the generic |
||||
choice error. Hence a first-class concept: ~80% parse semantics, ~20% display, |
||||
riding the existing rendering pipeline. |
||||
|
||||
## Approach |
||||
|
||||
- New per-argument key (spelling decided in the work; candidate -choiceunavailable |
||||
<list>). Entries must not also appear in -choices/-choicegroups (definition |
||||
error). Notes come from ordinary -choicelabels entries keyed by the unavailable |
||||
word. |
||||
- choiceword_match: unavailable entries join the prefix pool like reservelist |
||||
members, but a match landing on one returns a distinct indication so validation |
||||
can emit the tailored message ("'dict' is a recognised choice but not available |
||||
in this runtime - see its note") instead of the generic listed-values error. |
||||
The shared-resolver principle keeps parse, parse_status, the G-071 allocation |
||||
screen and the (G-051) doc walk automatically consistent. |
||||
- Display: render unavailable entries among the choices with their notes, |
||||
visually distinguished (group heading vs dim/warn styling vs marker - decided |
||||
in the work with display-cost in mind); excluded from value-in-effect and |
||||
goodarg marking; excluded from synopsis choice-literal alternates. |
||||
- tclcore adoption: the 'string is' define-time harvest computes |
||||
unavailable = curated_forward_list - live_classes (forward list: dict; |
||||
unicode excluded). The per-class virtual id loop extends to unavailable |
||||
classes using the static description so 'i string is dict' works as |
||||
documentation on 8.6. |
||||
- Parity: tclcoreparity.test gains a recorded exemption class for |
||||
prefix-collision strictness (model stricter than real 8.6 for 'di'-shaped |
||||
prefixes by user direction); full-word probes stay strict parity. |
||||
|
||||
## Notes |
||||
|
||||
- Related: G-040 (choiceword_match single-resolver principle and the reservelist |
||||
precedent), G-051 (doc walk consumes the resolver - achieved 2026-07-13), |
||||
G-071 (allocation screen consumes the resolver - achieved 2026-07-12), G-054 |
||||
archived (harvest + the parity pins this goal amends), G-055 (version-adaptive |
||||
definition technique; its workflow guidance gains the forward-class pattern |
||||
once proven here). |
||||
- The reject-message wording should steer the user: name the note/annotation and |
||||
the version boundary, not just "invalid". |
||||
@ -0,0 +1,86 @@
|
||||
# G-074 punk::args multiform ambiguity analysis: on-demand form-overlap detection with sanctioned-overlap annotation |
||||
|
||||
Status: proposed |
||||
Scope: src/modules/punk/args-999999.0a1.0.tm (analysis command, @form sanction key as decided in the work), src/modules/punk/args/moduledoc/tclcore-999999.0a1.0.tm (::after/::lseq as proving consumers, sanction adoption), src/tests/modules/punk/args/testsuites/args/ (new suite) |
||||
Goal: an on-demand analysis command (candidate spelling punk::args::formcheck <id>) reports, for a multiform definition, the form pairs an argument list could cleanly match simultaneously - a conservative static pass over the forms' leading slots and arity windows that flags discriminator-vs-permissive-type alignments (the 'lseq 1 count 5' class, where a literal/choice discriminator in one form aligns with a non-validating type like expr/any/string in another) and discriminator-free overlaps (the 'after cancel id|script' class) - with a definition-level annotation (spelling decided in the work, e.g @form key) that sanctions known/documented overlaps so they report as acknowledged rather than as findings, keeping the unsanctioned report actionable; the define hot path is untouched (analysis runs only on demand - interactive, test-time, and as a G-055 verification-gate step for regenerated multiform commands). |
||||
Acceptance: the analysis command reports the two known real cases from the G-041 closeout - ::lseq range/start_count overlap via the expr-typed end slot (unsanctioned finding: type-weakness class) and ::after cancelid/cancelscript (documented-overlap class) - and reports nothing for multiform definitions whose forms are fully discriminated (the parse withid/withdef pair, the forms.test afterish fixture); classifications distinguish at minimum type-weakness overlap (a discriminator aligned with a permissive type) from structural overlap (no discriminating slot exists); the sanction annotation silences (or downgrades to acknowledged) a listed form pair without affecting parse behaviour, is rejected at definition resolve when it names unknown forms, and is adopted for the after cancel pair in the tclcore moduledoc; the analysis is conservative in the documented direction (may miss deep ambiguities, must not false-alarm on discriminated forms - the miss/report boundary is documented with the slot model); running it performs no parse of user-supplied words and adds no work to define/resolve for definitions that never call it; a new testsuite covers the finding classes, the sanction, the unknown-form rejection and the no-finding cases; full punk::args suite passes. |
||||
|
||||
## Context |
||||
|
||||
User direction 2026-07-13 (during the G-041 closeout): "The encountered ambiguities |
||||
suggest weakness in the doc-blocked types - and perhaps requirement to evaluate for |
||||
ambiguities at define time to reject / warn"; the lint-time (on-demand) shape of this |
||||
goal was recommended and user-selected ("draft lint-time analysis"). |
||||
|
||||
The two ambiguities G-041's candidacy machinery surfaced on the tclcore models: |
||||
|
||||
- 'lseq 1 count 5' -> multipleformmatches range+start_count. Root cause is a |
||||
TYPE-WEAKNESS overlap: start_count's literalprefix(count) discriminator aligns |
||||
positionally with range's end slot typed number|expr, and -type expr performs no |
||||
validation, so the word 'count' satisfies both forms. Real lseq resolves it |
||||
semantically (start_count). The G-055 detail file carries the operand-typing |
||||
probes (2026-07-13): indexexpression is a poor lexical fit (under-accepts |
||||
1e2/2*3/sqrt(9)/bracketed exprs, over-accepts end/end-1); the precise |
||||
discriminator is expr SYNTAX validation, needing a non-evaluating parser |
||||
(G-069/G-070 territory); TIP 746 makes the operands number-only on 9.1+. |
||||
- 'after cancel someid' -> multipleformmatches cancelid+cancelscript. A GENUINE |
||||
documented overlap: 'after cancel id' vs 'after cancel script ?script?...' are |
||||
textually indistinguishable for one trailing word - real Tcl disambiguates by |
||||
the id's runtime shape. Correct in a doc-faithful model, but indistinguishable |
||||
from an authoring mistake without a sanction mechanism. |
||||
|
||||
Full ambiguity detection is language intersection between forms - not decidable in |
||||
general (types like regex-validated strings, unbounded -multiple tails) - but the |
||||
encountered classes are cheap to detect statically: walk the aligned leading slots |
||||
of each form pair over their overlapping total-arity windows and test whether every |
||||
slot pair is co-satisfiable, flagging pairs where a discriminator slot |
||||
(literal/literalprefix/restricted-choices) faces a permissive type (expr, any, |
||||
string, none-validating) and pairs with no discriminating slot at all. |
||||
|
||||
## Approach |
||||
|
||||
- Pairwise form analysis over the resolved spec (FORMS/ARG_INFO - no re-parse of |
||||
definitions): compute each form's value-arity window (VAL_MIN/VAL_MAX plus |
||||
optional-member clause arithmetic as in the allocator); for pairs with |
||||
intersecting windows, align leading slots and classify each aligned pair as |
||||
disjoint (two different literals; literal vs type that rejects it), |
||||
co-satisfiable-weak (discriminator vs permissive type), or co-satisfiable-equal |
||||
(same/overlapping permissive types). A pair with no disjoint slot in any aligned |
||||
position within the shared window is a finding: type-weakness if some slot is |
||||
discriminator-vs-permissive, structural otherwise. |
||||
- Choice/literal words screened via choiceword_match (the shared G-040 resolver) so |
||||
the static disjointness test cannot diverge from parse acceptance - the same |
||||
principle that kept G-071's allocation screen and G-051's doc walk honest. |
||||
- Options: forms whose option sets differ do not discriminate positionally (options |
||||
are order-free); first pass ignores opts for alignment (documented as part of the |
||||
conservative boundary) - the arity windows still gate. |
||||
- Sanction: an @form-level key (spelling decided in the work; candidates |
||||
-overlapallowed <formname list> on either member, or a definition-level |
||||
directive) recorded in the spec, validated at resolve (unknown form names error), |
||||
consumed only by the analysis (parse behaviour unchanged - G-041's |
||||
multipleformmatches error still fires; the sanction documents intent, it does |
||||
not pick a winner). |
||||
- Report shape: machine-parsable dict (per finding: forms, class, witness slot(s), |
||||
an example witness arglist shape where derivable, sanctioned flag) plus a human |
||||
summary - the dict is what the G-055 verification gate and test pins consume. |
||||
- Consumers: G-055's regeneration workflow gains a gate step "run formcheck on |
||||
every regenerated multiform command; new unsanctioned findings block acceptance"; |
||||
interactive use for definition authors debugging multipleformmatches errors. |
||||
|
||||
## Notes |
||||
|
||||
- Related: G-041 achieved 2026-07-13 (goals/archive/G-041-punkargs-form-matching.md) |
||||
- its candidacy machinery is the runtime oracle for a GIVEN arglist; this goal is |
||||
the static answer for ALL arglists, and its Progress section records both |
||||
motivating ambiguities. G-055 (verification-gate consumer; operand-typing probes |
||||
in its notes), G-072 (compound clause types change the slot model - if active |
||||
concurrently, coordinate the alignment walk), G-069/G-070 (a future expr |
||||
syntax-validating type would convert the lseq finding from type-weakness to |
||||
resolved - the analysis should then report the pair as discriminated). |
||||
- Define-time always-on checking was considered and rejected with the user's |
||||
concurrence: define/resolve is a hot path and most definitions are single-form; |
||||
on-demand covers the authoring and regeneration moments where findings are |
||||
actionable. A future opt-in (@dev-time strict mode) could call the same analysis. |
||||
- The 'after cancel' sanction is also the display question's anchor: a sanctioned |
||||
overlap is why the G-041 doc surface marks BOTH forms and shows the ambiguity |
||||
message - the sanction must not suppress that runtime honesty. |
||||
@ -0,0 +1,67 @@
|
||||
# G-075 punk::args (package) ids: working lookup and a user-facing package documentation surface |
||||
|
||||
Status: proposed |
||||
Scope: src/modules/punk/args-999999.0a1.0.tm (id lookup/update_definitions prefix handling, usage/arg_error rendering of package-level ids), src/modules/punk/ns-999999.0a1.0.tm (cmdhelp/help-system surface), src/modules/punk/mix/#modpod-templates-999999.0a1.0/templates/modules/template_module-0.0.4.tm (template block as touched/verified), src/tests/modules/punk/args/testsuites/args/ + src/tests/modules/punk/ns/testsuites/ns/ (new coverage) |
||||
Goal: the "(package)<pkgname>" ids that the module template (dev module.new) and the punk::args::moduledoc::* packages already declare become functional and user-visible - the id lookup path handles the (package) prefix (today punk::args::id_exists returns 0 for them and update_definitions emits an unqualified-ns warning, because only the (autodef) prefix is stripped when deriving a namespace from an id), a package-level id renders its @package -name/-help as a package documentation page through the usage machinery, and the interactive help surface reaches it ('i <pkgname>' presenting the package doc when <pkgname> resolves to a documented package rather than a command - exact surface decided in the work), making @package -help the natural home for per-package conventions and usage notes (e.g a moduledoc's harvest/provenance conventions). |
||||
Acceptance: punk::args::id_exists returns true for a registered "(package)<pkgname>" id after its namespace's scan (the update_definitions "received unqualified ns" warning no longer fires for (package)-prefixed lookups); punk::args::usage renders a (package) id via the @package directive's fields (render shape decided in the work - it has no arguments table unless the definition adds directives; at minimum -name and -help present cleanly in table and string renderers); the interactive surface is decided, documented and implemented ('i <pkgname>' or documented equivalent) with behaviour defined for the name-collision case (a namespace that is both a documented package and an ensemble command); the four moduledoc (package) ids (iocp, parser, tkcore, tzint) and at least one template-created module's id render through the new surface; the template's (package) block is verified as-written (or minimally corrected in step, with the template version bumped per its conventions); discoverability exists in some listed form (e.g ids matching (package)* via an existing query/listing command - decided in the work); new tests cover lookup, rendering and the interactive surface; full punk::args and punk::ns suites pass. |
||||
|
||||
## Context |
||||
|
||||
User context 2026-07-13: the (package) ids were intended somewhat as the surface for |
||||
package-level documentation/guidance but "aren't yet surfaced to the user"; they come |
||||
from the module template, so most modules created with 'dev module.new' carry one. |
||||
|
||||
Probed state (2026-07-13, tclsh 9.0.3): 45 modules under src/modules declare |
||||
@id -id "(package)<pkgname>" blocks with an @package -name/-help directive |
||||
(template_module-0.0.4.tm carries the canonical block). They are currently inert: |
||||
|
||||
- punk::args::id_exists "(package)punk::args::moduledoc::parser" returns 0 even with |
||||
the package loaded and registered. |
||||
- The lookup emits: warning: update_definitions received unqualified ns: |
||||
(package)punk::args::moduledoc - real_id derives a namespace from the id's |
||||
qualifiers, and only the (autodef) prefix is special-cased/stripped; the (package) |
||||
prefix flows into the namespace name. |
||||
- The resolver itself understands the @package directive (the spec dict carries a |
||||
package_info key alongside cmd_info/doc_info) - the gap is lookup and surfacing, |
||||
not definition storage. |
||||
|
||||
The immediate motivation is the guidance-space discussion (recorded in G-055 and the |
||||
punk::args 0.11.1 define doc): system-wide authoring mechanics now live in the |
||||
punk::args::define help, and per-package conventions (harvests, provenance, version |
||||
adaptivity of a moduledoc) belong with the package - @package -help is the declared |
||||
place, once reachable. |
||||
|
||||
## Approach |
||||
|
||||
- Lookup: extend the prefix handling where namespaces are derived from ids |
||||
(real_id/update_definitions) so "(package)" is recognised like "(autodef)" - |
||||
the namespace for scan purposes is the id minus the prefix. Verify tail-colon |
||||
handling ((package) ids name namespaces, not commands - the id has no command |
||||
tail). |
||||
- Rendering: punk::args::usage on a (package) id - drive the existing arg_error |
||||
machinery with package_info (title/name/help; no Arg table rows unless the |
||||
definition declares arguments). Check interaction with the synopsis section |
||||
(suppress or render a package-appropriate header). |
||||
- Interactive surface: punk::ns::cmdhelp fallback - when the query word is not a |
||||
command but matches a documented package/namespace, present the package page. |
||||
Collision rule (documented): a real command (e.g an ensemble named for the |
||||
namespace) wins; the package doc is reachable explicitly (spelling decided - |
||||
e.g 'i (package)foo::bar' at minimum). |
||||
- Discoverability: a listing of documented packages (existing id query/listing |
||||
machinery filtered on the prefix, or a punk::args::status extension). |
||||
- Template: verify the template block round-trips through the fixed path; the |
||||
moduledoc (package) blocks likewise (tclcore notably has none - adding one is |
||||
a natural but optional step recorded either way). |
||||
|
||||
## Notes |
||||
|
||||
- Related: G-055 (per-package conventions/provenance belong in @package -help once |
||||
surfaced; its workflow doc should point there when this lands), G-042 (subshell |
||||
help topics - adjacent help-surface work; keep the surfaces consistent), |
||||
G-017/G-068 (package-level documentation consumers). |
||||
- The punk::args::define help (0.11.1) documents registration styles and defspace |
||||
rules system-wide; this goal's surface is where the per-package layer of that |
||||
guidance becomes reachable. |
||||
- 45 declaring modules found 2026-07-13 via grep for '"(package)' under src/modules |
||||
(plus decktemplates vendor copies); count will drift - the acceptance names the |
||||
four moduledocs + one template-created module as the proving set, not all 45. |
||||
@ -0,0 +1,157 @@
|
||||
# G-039 Investigate the orphaned-shell one-core spin on a dead console |
||||
|
||||
Status: achieved 2026-07-12 |
||||
Scope: src/modules/punk/repl-999999.0a1.0.tm (console reader/event loop and EOF/error paths), src/modules/punk/console-999999.0a1.0.tm; investigation-first |
||||
Goal: the observed failure mode - an interactive punk902z left running after its hosting terminal/console went away spins roughly a full core indefinitely (observed 2026-07-08: a 37-minute orphan with a single hard-looping thread) - is reliably reproduced and root-caused, then fixed or mitigated so a shell whose console dies exits or reaches zero-CPU idle cleanly. |
||||
Acceptance: a documented procedure reproduces the spin on the current kit (e.g. launch an interactive shell in a terminal, then kill/close the hosting terminal or conhost), or the investigation records the attempts made and what evidence would reopen it; the spinning code path is identified (prime suspect: a console read/event loop treating a dead console's immediate EOF/error as retryable without backoff or termination - adjacent to the console-EOF restart path G-038 takes ownership of); after fix/mitigation, the same procedure shows the orphaned process exiting or settling at effectively zero CPU within a short grace period, with live-console interactive behaviour unchanged; the wedge-scoring hazard note (orphans polluting process-liveness checks in test harnesses) is updated to match the outcome. |
||||
|
||||
## Context |
||||
|
||||
Observation (2026-07-08, during the G-036 investigation): a punk902z process from an earlier |
||||
interactive session (started ~37 minutes prior; its hosting terminal presumed closed) was |
||||
found consuming CPU continuously - ~1550 CPU-seconds accumulating at roughly +0.5-1 |
||||
core-seconds per wall second, with exactly ONE thread in Running state (1546s of the total) |
||||
and every other thread idle in normal waits. The process had to be killed manually. It also |
||||
polluted the wedge-scoring checks of the day (process-liveness was being used as a hang |
||||
signal), which is how it was noticed. |
||||
|
||||
Not diagnosed at the time (killed to unblock the G-036 work); no dump was taken. The |
||||
suspicion: when the hosting console goes away, a console channel read/event path returns |
||||
immediately (EOF or error) and the surrounding loop retries without backoff or a |
||||
give-up-and-exit decision. Candidate sites: the repl reader loop, the EOF branch behaviour |
||||
when reopen/restart fails or loops (note tcl_interactive would be true for a real console, |
||||
enabling the `after 1 reopen_stdin` path - see the race notes in the G-038 detail file), or |
||||
punk::console query/read helpers. |
||||
|
||||
## Approach |
||||
|
||||
1. Reproduce: launch an interactive shell in a disposable terminal and kill the host |
||||
(close the tab/window; `Stop-Process` on the terminal; for classic conhost, killing |
||||
conhost.exe; also try Windows Terminal vs conhost - behaviour may differ). Watch the |
||||
orphan's CPU and thread states (`(Get-Process punk902z).Threads` sorted by |
||||
TotalProcessorTime). Try both idle-at-prompt and mid-command states at kill time. |
||||
2. If reproduced: procdump + WinDbgX scripted stack capture of the spinning thread (the |
||||
G-036 detail file documents the working tooling pipeline and pitfalls - WinDbgX process |
||||
name is DbgX.Shell, cdb is ACL-buried, `-z/-p -c -logo` scripting works). With the spin |
||||
thread's stack, map to the retry loop and decide fix: treat dead-console EOF/error as |
||||
terminal (exit per PUNK_PIPE_EOF-like policy), or add backoff + detection. |
||||
3. Coordinate with G-038: its caller-driven restart owns the console-EOF path; the dead |
||||
console case is the failure branch of the same decision point (reopen CONIN$ fails or |
||||
the console is gone entirely) and should be designed together. |
||||
|
||||
## Notes |
||||
|
||||
- Related: G-038 (console-EOF restart ownership, reopen_stdin race notes), G-036 (dump/ |
||||
debugging tooling pipeline), tcl86-console-parked-read prior art (different mechanism, |
||||
same neighbourhood). |
||||
- Harness hygiene (resolved 2026-07-12): kits were rebuilt with punk::repl 0.5.0 the same |
||||
day (plain-kit kill test re-verified: exit within 2.0s), so the stray-orphan precaution |
||||
applies only when deliberately testing pre-0.5.0 kits. |
||||
- Upstream: the root cause is a Tcl 9 core defect pair in win/tclWinConsole.c (see |
||||
Progress below): (a) ConsoleEventProc drops the error/EOF notification that |
||||
ConsoleSetupProc/ConsoleCheckProc generate for lastError != 0 (only ring-buffer data |
||||
produces a Tcl_NotifyChannel), so a script can never see a dead console via fileevent; |
||||
(b) ConsoleReaderThread's persistent-error branch (inputLen==0 && lastError!=0) wakes |
||||
CVs + NudgeWatchers and continues with no wait, busy-looping one core (and via the |
||||
wakeups a second) until the channel is closed. Both defects verified against a plain |
||||
stock tclsh 9.0.3 (repro script with a stdin fileevent + vwait: ~1.85 cores after |
||||
conhost kill, readable handler never fires) and the code is unchanged in the 9.1b1 |
||||
reference sources. A ready-to-file ticket draft with the standalone repro lives at |
||||
`TEMP_REFERENCE/tcl9-dead-console-spin-TICKET-DRAFT.md`. A kit-side Tcl source patch |
||||
is an alternative once G-005/G-006 build infrastructure exists; the script-level |
||||
watchdog stays valid regardless. |
||||
- Archived-goal references in this file: G-036 achieved 2026-07-08 (goals/archive/G-036-tcl9-udp-console-worker-wedge.md). |
||||
|
||||
## Progress |
||||
|
||||
### 2026-07-12 reproduced, root-caused, fixed in source (punk::repl 0.5.0) |
||||
|
||||
Reproduction procedure (reliable, first-try, current punk902z kit 2026-07-11 build): |
||||
|
||||
1. Launch the shell interactively under a dedicated classic conhost (isolates the kill |
||||
from the user's terminal): |
||||
`Start-Process -WindowStyle Hidden conhost.exe -ArgumentList 'C:\repo\jn\shellspy\bin\punk902z.exe'` |
||||
(optionally via cmd.exe with `> out.txt 2> err.txt` redirection to capture the trail; |
||||
`punk902z src` for dev modules). |
||||
2. Confirm the tree (`punk902z` is a child of the new conhost) and let it settle at the |
||||
idle prompt (CPU delta 0 over several seconds). |
||||
3. `Stop-Process -Force` the conhost pid. punk902z survives and immediately spins: |
||||
~1.7-1.9 cores, matching the 2026-07-08 observation (that one showed ~1 core; both |
||||
hot threads here are part of the same mechanism). Both idle-at-prompt kill state and |
||||
the redirected-stdio variant reproduce. |
||||
|
||||
Spin mechanism (dump: procdump64 -ma + `WinDbgX -z <dmp> -c "~*k 30; .logclose; q" -logo`, |
||||
per the G-036 tooling notes; mapped against TEMP_REFERENCE/tcl9/win/tclWinConsole.c): |
||||
|
||||
- Hot thread 1 = the Tcl console reader thread (stack: thread start -> ConsoleReaderThread |
||||
-> SetEvent storm). After the console dies, ConsoleDataAvailable returns -1 |
||||
(PeekConsoleInputW fails) which the caller treats as truthy, ReadConsoleChars fails |
||||
(observed error: broken pipe - ERROR_BROKEN_PIPE, not ERROR_INVALID_HANDLE) and |
||||
handleInfoPtr->lastError is set. From then on the reader thread's top-of-loop branch |
||||
(inputLen>0 || lastError!=0) runs every iteration - WakeAllConditionVariable + |
||||
NudgeWatchers(SetEvent/Tcl_ThreadAlert) + continue - with no sleep on that path, until |
||||
channel close drops numRefs to 1. lastError is never cleared because the script never |
||||
reads again (see next point). |
||||
- Hot thread 2 = the repl/main interp thread: each nudge alerts its notifier; |
||||
ConsoleSetupProc counts lastError as "readable" and sets max block time 0, and |
||||
ConsoleCheckProc queues a console event for it - but ConsoleEventProc only calls |
||||
Tcl_NotifyChannel when the ring buffer has DATA and ignores lastError entirely, so the |
||||
queued event is discarded and the armed readable fileevent (repl_handler) NEVER fires. |
||||
The event loop spins at zero block time queueing and discarding events. |
||||
- Script level is therefore completely blind: instrumented src-mode run confirmed |
||||
repl_handler never runs after console death (no entry, no EOF branch, no reopen_stdin, |
||||
no bgerror, vwait never returns), so nothing ever closes the channel and both spins are |
||||
permanent. The reopen_stdin restart loop suspected in the original goal statement is NOT |
||||
the mechanism - it never gets the chance to run. |
||||
|
||||
Fix (src/modules/punk/repl-999999.0a1.0.tm, punk::repl 0.5.0): |
||||
|
||||
- New `repl::console_watchdog` self-rescheduling liveness poll (default 5000ms, |
||||
`repl::console_watchdog_ms`), armed by repl::start before its vwait, only when platform |
||||
is windows AND the repl serves the process-default console AND the input channel's |
||||
`chan configure` dict has `-inputmode` (i.e. a tcl9 console channel; piped stdin, |
||||
foreign consoles and tcl 8.6 console channels never arm it). The probe is a read-only |
||||
`chan configure $inchan -inputmode` (live GetConsoleMode). On probe failure: disarm, |
||||
close the input channel (this is what stops the reader-thread spin - numRefs drops and |
||||
the thread exits), and set `::repl::done {eof <chan>}` so the repl finishes via the |
||||
normal eof path; app-punkshell's eof handling then finds CONIN$ unopenable ("broken |
||||
pipe") and exits cleanly (heuristic policy, exit 0). repl::start's post-vwait |
||||
`chan event $inchan readable {}` deregistration is now guarded for the |
||||
watchdog-closed-channel case (previously an unguarded call raised a traceback on this |
||||
path). Scheduling state per channel name in `repl::console_watchdog_afterids`; |
||||
cancelled after vwait by the arming frame. |
||||
|
||||
Verification (2026-07-12, punk902z kit 2026-07-11 in `src` mode - dev modules; Tcl 9.0.2 |
||||
kit runtime): |
||||
|
||||
- Same kill procedure post-fix: watchdog message on stderr, orphan EXITED 1.5s after the |
||||
conhost kill (worst case is ~watchdog interval + teardown), no traceback. Remaining |
||||
teardown noise (disableRaw warning, codethread tsv warning) is cosmetic, printed to an |
||||
already-dead console in the real scenario. |
||||
- Live-console soak: 25s at the idle prompt (>=4 watchdog probes) - process alive, CPU |
||||
delta 0, prompt intact; no spurious trigger. |
||||
- Piped stdin (`'set ::x 1' | punk902z src` with PUNK_PIPE_EOF=exit): unaffected, exit 0 |
||||
(watchdog not armed for pipes). |
||||
- runtests (native tclsh 9.0.3): repl consolebackends.test 3/3 pass; punk::console suites |
||||
(console/*.test) 88 pass / 1 skip / 0 fail. |
||||
- `tclsh src/make.tcl modules` builds clean (provenance warnings = expected dirty tree). |
||||
|
||||
### 2026-07-12 acceptance closed (kits rebuilt, user verification) |
||||
|
||||
All remaining items resolved: |
||||
|
||||
- Kits rebuilt 2026-07-12 with punk::repl 0.5.0 (punkbi, punk91, punk902z). Plain-kit |
||||
kill procedure re-verified on the rebuilt punk902z: orphan exited 2.0s after the |
||||
conhost kill via the clean eof path (watchdog message, app-punkshell "no console |
||||
available", exit 0), no traceback. |
||||
- User verified interactive sessions on the rebuilt kits (punkbi, punk91, punk902z, |
||||
punk::repl confirmed at 0.5.0): behaviour unchanged with the 5s probe - "all seems |
||||
well". |
||||
- Tcl 8.6 (punksys) confirmed OUT OF SCOPE by the user 2026-07-12: different console |
||||
driver, watchdog deliberately not armed there; reopen as a new goal if an 8.6 orphan |
||||
spin is ever observed. |
||||
- Wedge-scoring hazard note updated (see Notes - precaution now applies only to |
||||
pre-0.5.0 kits). |
||||
- Root cause verified independent of punkshell with plain tclsh 9.0.3; upstream ticket |
||||
draft written to `TEMP_REFERENCE/tcl9-dead-console-spin-TICKET-DRAFT.md` (filing is a |
||||
user decision, not gating this goal). |
||||
@ -0,0 +1,221 @@
|
||||
# G-041 punk::args multi-form matching: automated form selection for parsing and documentation |
||||
|
||||
Status: achieved 2026-07-13 |
||||
Scope: src/modules/punk/args-999999.0a1.0.tm (parse form selection, arg_error/usage form marking), src/modules/punk/ns-999999.0a1.0.tm (cmdhelp/synopsis closest-form indication), src/tests/modules/punk/args/testsuites/ |
||||
Goal: for multi-form definitions punk::args determines which form(s) an argument list matches - parse without -form attempts all permitted forms instead of effectively form 0, -form accepts the documented list-of-forms restriction, and the documentation surface indicates the match ('i after cancel <id>' presents the cancel form; 's after cancel someid' marks the closest synopsis) - with explicit single-form restriction retained for callers that require it. |
||||
Acceptance: parsing a multiform definition (after-like fixture) without -form succeeds when the args match exactly one form (the pinned GAP tests in forms.test flip to auto-selected results); an argument list matching no form (or several) produces an error naming the candidate forms rather than a form-0 type error; -form with a list of form names/indices restricts parsing to that subset (currently an 'Expected int 0-N or one of ...' error); 'i <cmd> <args...>' and synopsis output indicate the best-matching form(s) for supplied args; explicit -form <single-name-or-index> behaviour is unchanged; the full punk::args and punk::ns suites pass. |
||||
|
||||
## Context |
||||
|
||||
Multi-form definitions (@form directive) describe commands whose argument sets are |
||||
orthogonal - the motivating example is the Tcl builtin `after`, documented with several |
||||
forms (`after ms ?script...?`, `after cancel id`, `after idle script...`, ...). The |
||||
definition and display side works: form_names are recorded, `punk::ns::synopsis` renders |
||||
every form (## FORM n headers), and `punk::args::parse -form <name-or-index>` parses |
||||
against an explicitly chosen form. |
||||
|
||||
What is missing (characterized 2026-07-08 with an after-like fixture, probe results |
||||
pinned as GAP tests in src/tests/modules/punk/args/testsuites/args/forms.test): |
||||
|
||||
- `punk::args::parse` without -form (default `*`) does NOT attempt the other forms: |
||||
arguments that match only a non-zero form (e.g. `cancel after#1`) fail with form 0's |
||||
type/arity error ("Bad number of values ... Not enough remaining values", the leading |
||||
word rejected by form 0's int type). Effectively `*` parses form 0 only. |
||||
- `-form` is documented as "Restrict parsing to the set of forms listed" and typed as a |
||||
list, but a list of two form names errors with "invalid -form value ... Expected int |
||||
0-N or one of '<names>'" - only a single name or index is accepted. |
||||
- Documentation surface: `i after cancel <id>` renders help for the default form rather |
||||
than determining that the cancel form is the closest match, and `s after cancel someid` |
||||
lists all synopses without marking the matching one. |
||||
- Internal note: punk::args::parse's own definition is multiform (withid/withdef, |
||||
discriminated by literal types) but the proc hand-parses its arguments on the happy |
||||
path, so the multiform selection machinery is not exercised internally either. |
||||
|
||||
User direction (2026-07-08): there are times when restricting to a particular form is |
||||
appropriate and times when automated selection is better - both must remain expressible. |
||||
This is an advanced improvement, deferred; the immediate prerequisite work was making the |
||||
punk::args test suite comprehensive so the current behaviour is pinned before any |
||||
form-selection logic lands. |
||||
|
||||
## Approach (sketch - to be refined when activated) |
||||
|
||||
1. Form candidacy: for -form * (or a list), attempt each permitted form's parse; classify |
||||
outcomes (clean match / match-with-defaults / type-or-arity failure). Exactly one |
||||
clean match -> auto-select. None or several -> error naming candidate forms (and for |
||||
'several', possibly a deterministic preference rule - e.g. first-declared - recorded |
||||
when decided). |
||||
2. Literal/leading-word discrimination as a fast path: many multiform commands (after, |
||||
parse itself) discriminate on a leading literal - a pre-pass on literal/literalprefix |
||||
leader types can pick the form without full parse attempts. |
||||
3. -form list support: accept a subset of names/indices as documented; single value |
||||
behaviour unchanged. |
||||
4. Documentation surface: cmdhelp/arg_error accept the supplied trailing args (cmdhelp |
||||
already parses them for goodargs marking) and use the same candidacy logic to pick or |
||||
rank forms; synopsis display marks the matching form(s). |
||||
5. Error-message quality: a no-form-matches error should show per-form failure reasons |
||||
compactly rather than only form 0's. |
||||
|
||||
Downstream consumer note (2026-07-08, G-044): the interactive command-completion/hinting |
||||
goal consumes form candidacy on PARTIAL argument lists - as the user types, the hinting |
||||
display wants "which forms are still compatible with the words so far" - so the candidacy |
||||
mechanism should be exposed as an API that accepts an incomplete argument list and returns |
||||
per-form compatibility (not only an all-or-nothing full parse). Design for that consumer |
||||
when shaping step 1. |
||||
|
||||
## Progress |
||||
|
||||
- 2026-07-13 increment 1 (engine - punk::args 0.10.0, project 0.12.14): multi-form |
||||
candidacy implemented in get_dict. The single-form parse body was extracted verbatim |
||||
to private::get_dict_form (no caller-frame use inside - verified; resolve still runs |
||||
in the caller's context before selection). -form * / multi-form selections attempt |
||||
each permitted form: exactly one clean match auto-selects; none raises 'noformmatch' |
||||
naming each candidate form's first-line failure; several raise 'multipleformmatches' |
||||
naming the forms. -form list support landed across get_dict/parse/parse_status/ |
||||
arg_error via the shared private::form_selection resolver (supplied order preserved; |
||||
arg_error renders the argument table for the first listed form and marks all listed |
||||
forms' synopsis entries). Results gain 'form' (+'formstatus' when candidacy ran); |
||||
parse_status 'form' is the matched/best-candidate form and its new 'formstatus' key |
||||
is the per-form compatibility surface designed for the G-044 hinting consumer. |
||||
DECISIONS RECORDED: |
||||
- Several clean matches ERROR rather than silently preferring first-declared (the |
||||
acceptance wording; a silent pick could validate against an unintended form and |
||||
return a wrong values shape - callers with a preference pass -form). |
||||
- Candidate ranking (display/status only - candidacy always from real parse |
||||
attempts, no literal fast-path skipping): leading-literal affinity of each form |
||||
with the supplied words (agreeing literal(x)/literalprefix(x) leading run scores |
||||
up, first disagreement disqualifies), then incomplete before invalid, then |
||||
declaration order. Known limits noted in form_literal_affinity: no alignment |
||||
through received options; choice-discriminated forms score 0 (refinable). |
||||
- Literal-discrimination pre-pass as a parse fast path (approach step 2) deferred |
||||
as a perf refinement: correctness must come from real parse attempts, and the |
||||
affinity heuristic's word alignment is not exact through options. |
||||
Tests: forms.test GAPs flipped (forms_parse_autoselect, |
||||
forms_parse_formlist_restriction) + noformmatch ranking/errorcode, |
||||
multipleformmatches, shared-prologue arity discrimination, formstatus coverage. |
||||
punk::args 198/199 (1 pre-existing skip), punk::ns 53/53, punk::lib 35/35. |
||||
Remaining after increment 1: doc-surface increment (cmdhelp best-form presentation |
||||
via parse_status form key - cmdhelp's -form default was still 0; synopsis |
||||
closest-form marking), live tclcore verification ('lseq 3', switch block form), |
||||
@form -synopsis override GAP (adjacent - decide in doc-surface increment). |
||||
- 2026-07-13 increment 2 (doc surface - punk::args 0.11.0, punk::ns 0.5.0, project |
||||
0.12.15): cmdhelp -form defaults to * and renders the advisory parse's matched/ |
||||
best-candidate form at both render sites ('i after cancel <id>' presents the cancel |
||||
form's table; noformmatch passes the ranked candidates and multipleformmatches the |
||||
matching forms to arg_error so all are marked in the synopsis block). |
||||
punk::ns::synopsis underlines the form(s) trailing words match ('s after cancel |
||||
someid' marks both cancel forms; 's lseq 0 10 2' marks the range form) - ordinal |
||||
line-to-form mapping, skipped under alias-currying excess or an explicit -form. |
||||
The adjacent @form -synopsis override GAP was fixed and flipped |
||||
(forms_form_synopsis_override_rendered): punk::args::synopsis now renders the |
||||
documented override in full and summary modes. |
||||
Ranking refinement: form_literal_affinity extended to RESTRICTED-choice |
||||
discriminators via choiceword_match (shared G-040 resolver) - the tclcore models |
||||
express subcommand words as -choices (after's cancel/idle/info), so 'after cancel' |
||||
ranks the cancel forms first instead of declaration-order fallback. |
||||
REAL-MODEL FINDINGS (live tclcore verification, punk902z + tclsh probes): |
||||
- 'lseq 3' -> count form, 'lseq 0 10 2' -> range form, switch separate/block both |
||||
auto-select, 'i lseq 0 10 2' renders the range form info-scheme with its synopsis |
||||
line underlined. |
||||
- 'after cancel someid' is GENUINELY ambiguous in the doc-faithful model (cancelid |
||||
'after cancel id' vs cancelscript 'after cancel script ?script?...') - real Tcl |
||||
disambiguates semantically by the id's shape, a textual model cannot. Behaviour: |
||||
multipleformmatches naming both, first candidate's table, both synopsis lines |
||||
marked. Accepted as correct for a doc-faithful model. |
||||
- 'lseq 1 count 5' is ambiguous (range + start_count) ONLY because the model's |
||||
expr-typed operands accept the word 'count' as an end value - the -type expr |
||||
non-validation already recorded for G-055; noted there as a model-tightening |
||||
candidate (TIP 746 removes expr operand behaviour in 9.1 anyway). |
||||
Suites: punk::args 198/199 (1 pre-existing skip), punk::ns 57/57, full source tree |
||||
836 total / 822 passed / 13 skipped / 1 failed = exec-14.3 (the known baseline). |
||||
- 2026-07-13 ACHIEVED - acceptance review against each clause: |
||||
- Multiform parse without -form succeeds on a unique clean match: the forms.test |
||||
GAP pins flipped to auto-selected results (forms_parse_autoselect - ms/cancel/ |
||||
idle each selected from the words alone; forms_parse_autoselect_shared_prologue |
||||
covers arity-discriminated forms sharing a leader block). |
||||
- No-match/several-match errors name the candidate forms instead of a form-0 type |
||||
error: noformmatch carries every candidate's first-line failure ranked |
||||
best-first (forms_parse_noformmatch) and multipleformmatches names the matching |
||||
forms (forms_parse_multipleformmatches) - the 'several' outcome is an error by |
||||
recorded decision (no silent first-declared preference). |
||||
- -form list restriction: a list of names/indices restricts candidacy to the |
||||
subset across get_dict/parse/parse_status/arg_error |
||||
(forms_parse_formlist_restriction; unrecognised elements keep the invalid |
||||
-form error). |
||||
- Doc surface indicates the best-matching form(s): 'i after cancel <id>' presents |
||||
the cancel form's argument table (cmdhelp_multiform_autoselected_form_presented, |
||||
cmdhelp_multiform_noformmatch_best_candidate) and 's after cancel someid' |
||||
underlines the matching synopsis line(s) |
||||
(synopsis_multiform_marks_matching_form/_unmarked_without_args); live-verified |
||||
against the tclcore models on punk902z ('i lseq 0 10 2' range form underlined |
||||
info-scheme; 'i after cancel someid' ambiguity named with both cancel forms |
||||
marked). |
||||
- Explicit single -form behaviour unchanged: the pre-existing explicit-form tests |
||||
(forms_parse_explicit_form_by_name/_by_index) pass unmodified. |
||||
- Full punk::args and punk::ns suites pass (198/199 with 1 pre-existing skip; |
||||
57/57); full source tree 822/836 with the exec-14.3 baseline as the only |
||||
failure. |
||||
Adjacent extra: the @form -synopsis override GAP flipped |
||||
(forms_form_synopsis_override_rendered). The G-044 candidacy-API design note is |
||||
satisfied by parse_status's formstatus key (per-form compatibility on partial |
||||
arglists). Deferred as refinements (not acceptance): literal-discrimination |
||||
fast path for parse performance; affinity alignment through received options; |
||||
punk::args::synopsis -form list support (single-form restriction retained there). |
||||
|
||||
## Alternatives considered |
||||
|
||||
- Documenting alternate forms as separate subhelp choiceinfo entries (the trace-style |
||||
pattern) - works only when a literal subcommand word exists and fragments a command's |
||||
documentation into pseudo-subcommands; rejected as the general answer. |
||||
- Requiring callers to always pass -form - status quo for parsing, but unusable for the |
||||
interactive doc surface ('i after cancel <id>') where the user supplies args, not form |
||||
names. |
||||
|
||||
## Notes |
||||
|
||||
- Probe script: scratchpad formprobe.tcl (2026-07-08); findings encoded in |
||||
src/tests/modules/punk/args/testsuites/args/forms.test (GAP-labelled tests reference |
||||
this goal and flip when it is implemented). |
||||
- Related: G-040 (choice aliasing/doc-lookup parity - same "parser and doc surface must |
||||
agree" principle); punk::args::parse -cache interaction with per-form attempts will |
||||
need review (cache key already includes selected form). |
||||
- Incidental find during the same probing: an uncommented debug `puts stderr |
||||
">>>_get_dict_can_assign_value NOT alloc_ok..."` fired on every failed clause type |
||||
assignment (punk/args ~l.6186; its happy-path twin was already commented). Fixed |
||||
2026-07-08 (punk::args 0.2.3) - noted here because form-attempt logic will exercise |
||||
that failure path heavily. |
||||
- 2026-07-12 prework probes (user concerns raised before activation - real-builtin |
||||
evidence beyond the after fixture; results from the punk902z kit, Tcl 9.0.2): |
||||
- Auto-selection gaps confirmed on real commands: 'lseq 3' (count form), |
||||
'lseq 1 count 5 ?by 2?' (start_count form) and 'switch abc {a {} default {}}' |
||||
including options+block ('switch -regexp -matchvar m abc {...}') all fail under |
||||
default form-0 parsing but parse correctly with an explicit -form. The per-form |
||||
models are doc-faithful: lseq.n's three synopsis lines map 1:1 to the modelled |
||||
range/start_count/count forms; switch separate/block both verified per-form. |
||||
- PREREQUISITE independent of form selection: the value allocator mishandles an |
||||
optional single-word choice value followed by a required value plus a trailing |
||||
optional-member clause. lseq range-form arglists WITHOUT the ../to noise word |
||||
but WITH a step ('0 10 2', '0 10 by 2', '1 5 by 0') fail IN the range form (the |
||||
error blames '..|to'), while '0 to 10 2' and '0 .. 10 by 2' parse fine. No other |
||||
lseq form accepts those arglists, so form auto-selection alone could not make |
||||
'i lseq 0 10 2' work. The allocator prerequisite became G-071 and was FIXED |
||||
the same day (achieved 2026-07-12, punk::args 0.9.0 allocation choice screen - |
||||
see goals/archive/G-071-punkargs-optional-allocation.md): the lseq matrix now |
||||
parses in-form, leaving form auto-selection as this goal's remaining gap for |
||||
the 'lseq 3' / switch-block-form cases. ::if's noise-word clauses |
||||
('?literal(then)?' mid-clause, '?literal(else)?' clause-leading) parse correctly |
||||
in the same probes, so the failure is specific to the optional standalone value |
||||
+ trailing optional-member clause combination, not optional clause members |
||||
generally. |
||||
- CORRECTED (G-071 closeout): parse_status DOES accept -form - positioned |
||||
before the withid/withdef tail per its documented synopsis; the probe had |
||||
passed it trailing. Behaviour pinned in allocation.test |
||||
(parsestatus_form_option_order). The candidacy API and the G-044 consumer |
||||
can build on the existing surface (list-of-forms support remains this |
||||
goal's -form list item, as for parse). |
||||
- Over-acceptance divergences (model valid where the real command errors) recorded |
||||
for the modelability list in G-055's detail file - not this goal's scope. |
||||
- Incidental fix during probing: another unconditional debug puts on the clause |
||||
type-check path (private::get_dict_can_assign_value "checking tp ...", fixed as |
||||
punk::args 0.8.2) - companion to the 0.2.3 find noted above; form-attempt logic |
||||
will exercise these paths heavily. |
||||
- Archived-goal references in this file: G-040 achieved 2026-07-08 (goals/archive/G-040-punkargs-choicealiases.md). |
||||
@ -0,0 +1,263 @@
|
||||
# G-045 punk::args definition authoring ergonomics: record continuation, @cmd unindented fields, constructed-definition normalization |
||||
|
||||
Status: achieved 2026-07-12 |
||||
Scope: src/modules/punk/args-999999.0a1.0.tm (record parsing in resolve, tstr interplay, arg_error @cmd rendering), src/tests/modules/punk/args/testsuites/ (rendering.test/defquoting.test as the safety net), src/modules/punk-999999.0a1.0.tm (::punk::helptopic::define_docs de-hacked as the consumer proof) |
||||
Goal: authoring punk::args definitions no longer requires backslash line-continuations or ad-hoc workarounds for multi-line records and constructed definitions - a parser-recognised record-continuation mechanism (candidate: an unquoted trailing -& token, with the detail file recording the collision analysis and the element-count disambiguation alternative), -unindentedfields honoured for @cmd fields, and constructed (string-built) definitions able to opt into the same whole-block indent normalization file-style definitions get - with the container quoting rules (braced=literal, quoted=Tcl backslash semantics, \$\{ escape) promoted from defquoting.test into the define documentation. |
||||
Acceptance: a definition using the chosen record-continuation mechanism parses identically to its backslash-continuation equivalent (existing definitions unchanged - continuation is additive), with the token's collision rules documented and an escape/rejection story for values that legitimately match it; @cmd -help/-summary honour -unindentedfields (the rendering.test GAP rendering_unindentedfields_cmd_help_GAP flips to aligned); a constructed definition can request whole-block normalization so embedded continuation indentation behaves as in file-style definitions (the rendering_constructed_def_indent_characterization expectations updated to the chosen semantics), and ::punk::helptopic::define_docs drops its manual pre-normalization to prove it; the quoting rules from defquoting.test appear in the punk::args::define -help documentation; the full punk::args suite (128 tests incl. the rendering invariants: nesting independence, relative-indent preservation) passes with GAP tests flipped, none weakened. |
||||
|
||||
## Context |
||||
|
||||
Drafted 2026-07-09 after the tests-first characterization pass (user direction: implement |
||||
tests before changes in this area). The pain points, in the user's framing: |
||||
|
||||
- The define-block syntax deliberately avoids Tcl dict syntax for human readability, which |
||||
led to backslash line-continuations and a complex interplay between record parsing in |
||||
punk::args::resolve and tstr (especially ${...} expansions keeping whitespace aligned |
||||
during multiline substitutions). |
||||
- Constructed (string-built) definitions - ::punk::helptopic::define_docs and |
||||
punk::args::ensemble_subcommands_definition are the in-tree examples - have poor |
||||
ergonomics for getting indentation right: unlike file-style definitions they receive no |
||||
whole-block indent normalization (proven: rendering.test |
||||
rendering_constructed_def_indent_characterization - continuations at exactly 4 spaces |
||||
align, other depths leak), so builders pre-normalize by hand (undent/trim hacks, |
||||
\n-relative authoring). |
||||
- A key property of normal definitions that must be preserved: they do not depend on the |
||||
indentation of the code block they appear in (proven and pinned: |
||||
rendering_nesting_independence_plain / _tstr). |
||||
- The user's experiment used an ad-hoc `--` flag followed by a braced line-spanning empty |
||||
string as a de-facto record continuation - functional but not designed-for, and |
||||
unsatisfying. |
||||
|
||||
## The -& record-continuation candidate (implemented 2026-07-12 - see Progress increment 3) |
||||
|
||||
The user's proposal: a special token the record parser recognises as record continuation, |
||||
e.g. an unquoted `-&` at end of line. Design considerations recorded for implementation: |
||||
|
||||
- Collision risk is low but not zero (a value or flag legitimately spelled -&). Candidate |
||||
rules: only an UNQUOTED trailing `-&` at end of a record line counts; and/or use the |
||||
element count of the assembled record to disambiguate (an odd element count where pairs |
||||
are expected implies the trailing token is a continuation marker, not a value). The |
||||
detail decision must include an escape story for a literal trailing -& value (bracing it |
||||
suffices if 'unquoted' is part of the rule). |
||||
- Detection must happen in the record parser (resolve) before/alongside line splitting - |
||||
the same place backslash-continuation assembly effectively happens via Tcl's own parse |
||||
of braced blocks today. Unlike backslash-newline, -& survives inside braced blocks of |
||||
CONSTRUCTED definitions too - which is precisely why it helps: string-built defs cannot |
||||
use backslash-newline ergonomically (it is consumed by the building code's own quoting). |
||||
|
||||
Decision as implemented (punk::args 0.7.0): |
||||
|
||||
- Rule: unquoted trailing `-&` only - the token must be a bare word preceded by |
||||
whitespace (or be the whole line) and be the last element on the line; trailing |
||||
whitespace after it is tolerated (deliberately more forgiving than raw |
||||
backslash-newline, whose invisible-trailing-whitespace failure is a classic trap). |
||||
- Escape story: brace or double-quote a literal trailing -& value ({-&}) - the raw line |
||||
then ends with the closing delimiter and never matches. A -& mid-line, as a word |
||||
suffix (abc-&), or on a line inside a still-open braced/quoted value is data. |
||||
- Element-count disambiguation: NOT implemented. The positional rule is deterministic |
||||
and locally decidable per line; element counting would require assembling and |
||||
list-parsing the record first (fragile against in-progress records) and gives |
||||
confusing action-at-a-distance when a distant key/value slips the count. Rejected |
||||
rather than deferred - the escape story covers the residual collision (a literal |
||||
trailing -default -& must be braced). |
||||
- Assembly semantics: byte-identity with the backslash equivalent. Key finding: for |
||||
braced (file-style) definitions Tcl itself collapses backslash-newline plus following |
||||
whitespace to a single space BEFORE the text reaches the record splitter - so -& is |
||||
implemented the same way (token dropped, single-space join, next line's leading |
||||
whitespace collapsed), making the assembled record byte-identical to its |
||||
backslash-continued twin. Proven by parse-result and rendered-table equality in |
||||
recordcontinuation.test. |
||||
- Reserved-word consequence: an argument cannot be NAMED -& via a record line ending in |
||||
the bare token (e.g. a lone '-& ' line would read as a continuation). Any such need is |
||||
met by bracing. Considered acceptable and documented here rather than guarded in code. |
||||
|
||||
## @cmd -unindentedfields |
||||
|
||||
Accepted but ignored today (arg_error's cmd_info rendering carries an '#unindentedfields ?' |
||||
todo; pinned by rendering_unindentedfields_cmd_help_GAP: left-margin-authored @cmd help |
||||
renders its first line +4 vs continuations). Honouring it makes left-margin authoring |
||||
available for @cmd -help/-summary as it already is for argument -help and -choicelabels. |
||||
|
||||
## Constructed-definition normalization |
||||
|
||||
Options to decide at implementation (record the choice): |
||||
- an explicit define option/directive requesting whole-block undent of the supplied text |
||||
- or automatic common-prefix normalization when a definition arrives as a single text |
||||
block (risk: changing existing constructed defs' rendering - the characterization test |
||||
documents current expectations to update deliberately) |
||||
::punk::helptopic::define_docs currently pre-normalizes by hand (undent/trim + \n-relative |
||||
help strings + -unindentedfields) - it converts to the new mechanism as the proof, and the |
||||
user's uncommitted experimental hack in punk.tm is superseded. |
||||
|
||||
## Quoting documentation |
||||
|
||||
defquoting.test pinned the container rules (braced values fully literal - $, [], two-char |
||||
\n, bare backslashes; quoted values get Tcl backslash semantics - \n newline, \\ -> \ - |
||||
with $/[] still literal; \$\{...\} renders a literal ${...} in tstr-processed blocks). |
||||
Promote these rules into the punk::args::define -help documentation so documenters (e.g. |
||||
punk::imap4's {\Deleted}/{$MDNSent} choice values) have them stated, not just tested. |
||||
|
||||
## Alternatives considered |
||||
|
||||
- Keeping backslash continuations + manual pre-normalization as the documented way - |
||||
rejected: the friction is proven by two in-tree constructed-def workarounds and the |
||||
user's punk.tm experiments. |
||||
- Tcl dict syntax for definitions - rejected long ago by design (readability). |
||||
- The ad-hoc `--` + braced-gap trick as the blessed continuation - rejected: exploits |
||||
parser behaviour that was never a contract. |
||||
|
||||
## Notes |
||||
|
||||
- Safety net: src/tests/modules/punk/args/testsuites/args/rendering.test (15 tests: |
||||
nesting independence, relative-indent preservation, unindentedfields, constructed-def |
||||
characterization, multiline tstr insertions, @dynamic) and defquoting.test (container |
||||
quoting rules). GAPs to flip here: rendering_unindentedfields_cmd_help_GAP, |
||||
rendering_constructed_def_indent_characterization (expectations updated to chosen |
||||
semantics). |
||||
- Related: G-046 (deferred -help resolution and rendering fixes - same code |
||||
neighbourhood; sequencing between the two goals is free but they touch resolve |
||||
together), G-044 (completion consumes the documentation corpus these ergonomics grow). |
||||
- Session records 2026-07-09: probe scripts renderprobe*.tcl (scratchpad), the user's |
||||
experimental define_docs hack (uncommitted punk.tm working-tree diff at the time of |
||||
drafting - superseded by this goal when implemented). |
||||
- Archived-goal references in this file: G-046 achieved 2026-07-10 (goals/archive/G-046-punkargs-deferred-help-and-fixes.md). |
||||
|
||||
## Progress |
||||
|
||||
### 2026-07-12 increment 1: @cmd -unindentedfields honoured (punk::args 0.6.1) |
||||
|
||||
- arg_error's cmd-help display transform (undent " "+help, max 4 - the |
||||
'#unindentedfields ?' todo site) is now gated by "-help" membership in the @cmd |
||||
line's -unindentedfields list, mirroring the existing per-argument gate. The |
||||
single transform site feeds both the table and string renderers. |
||||
- rendering_unindentedfields_cmd_help_GAP flipped to |
||||
rendering_unindentedfields_cmd_help (expects aligned; both renderers measured). |
||||
- @cmd -summary: verified no renderer applies an indent transform to -summary (it |
||||
is only used inline in synopsis "# ..." lines), so -unindentedfields membership |
||||
for -summary is accepted and vacuously honoured - nothing to gate. This |
||||
satisfies the acceptance's "-help/-summary honour" clause for -summary. |
||||
- Blast radius checked before the change: no in-tree definition sets |
||||
-unindentedfields on a @cmd line, so no existing rendering changed. |
||||
- define doc for -unindentedfields now states where the option is valid |
||||
(argument lines and the @cmd directive's -help). |
||||
- Verified (tclsh 9.0.3): full punk::args suite 175 pass / 1 pre-existing skip / |
||||
0 fail; punk::ns suite 53/53 (arg_error consumer); make.tcl modules builds |
||||
clean. |
||||
|
||||
### 2026-07-12 increment 2: 'i help' alignment via -unindentedfields (punk 0.2.4) |
||||
|
||||
- ::punk::helptopic::define_docs now authors its help text at the left margin |
||||
and declares -unindentedfields {-help} on the generated @cmd line (using |
||||
increment 1's gate) and on the topic argument line (a gate that existed all |
||||
along but was never applied here). Verified in punk902z src: the Description |
||||
block's +12 continuation leak and the topic Help first-line +4 are both gone; |
||||
'i help' renders flush end-to-end. punk::ns suite 53/53; make.tcl modules |
||||
clean. |
||||
- Decision (user, 2026-07-12): when the constructed-def normalization mechanism |
||||
lands, define_docs converts from its interim left-margin authoring to |
||||
indented-plus-normalized authoring - it remains the acceptance's consumer |
||||
proof as written. Left-margin authoring via -unindentedfields stays a |
||||
supported style proven by tests (rendering_unindentedfields_arg_help, |
||||
rendering_unindentedfields_cmd_help), but it is not the preferred style for |
||||
define_docs itself. |
||||
|
||||
### 2026-07-12 increment 3: -& record continuation (punk::args 0.7.0) |
||||
|
||||
- Implemented in private::split_definition_records per the decision recorded in |
||||
"The -& record-continuation candidate" section above (unquoted trailing token, |
||||
brace escape, element-count alternative rejected, byte-identical assembly to |
||||
the backslash equivalent). |
||||
- First implementation attempt kept a literal backslash-newline in the assembled |
||||
record and failed: braced definitions never contain backslash-newline by the |
||||
time they reach the splitter (Tcl pre-collapses it), and downstream record |
||||
handling assumes the collapsed shape. The collapse-to-single-space rewrite is |
||||
the correct equivalence and is what landed. |
||||
- New testsuite recordcontinuation.test (6 tests): backslash-twin parse+render |
||||
equality (same-length ids so table geometry matches), constructed-def chaining |
||||
incl. into a multi-line quoted value, braced {-&} escape, mid-line -&, |
||||
word-suffix abc-&, -& inside still-open braces. |
||||
- define doc documents -& alongside the backslash continuation paragraph. |
||||
- Verified (tclsh 9.0.3): full punk::args suite 181 pass / 1 pre-existing skip / |
||||
0 fail; full source-tree suite (splitter regression sweep) 815 total, 801 |
||||
pass, 13 skip, 1 fail = exec-14.3 only (the known pre-existing core-test |
||||
baseline) - zero regressions. |
||||
- tclparser considered and rejected for the splitter (user question 2026-07-12): |
||||
parse command mis-tokenizes the dialect's legitimate unbalanced-bracket ANSI |
||||
data inside quoted values (info complete gets a throwaway ansistripped copy; |
||||
parse would need strip+range-remapping), the punk semantics (base-indent line |
||||
trimming, -&) sit outside Tcl's grammar either way, and a binary dependency in |
||||
punk::args inverts G-004 for the module that must load everywhere. tclparser |
||||
remains right for real-script analysis (punk::lib/G-019). Candidate flagged: |
||||
dev-time lint cross-checking splitter boundaries against parse where the |
||||
binary is available. |
||||
|
||||
### 2026-07-12 increment 4: @normalize constructed-def normalization (punk::args 0.8.0, punk 0.2.5) |
||||
|
||||
- Mechanism (user-approved 2026-07-12): bare @normalize directive (like @dynamic; |
||||
options are an error), detected as a resolve pre-pass over the split records. |
||||
Semantics user-confirmed as re-base-to-+4; narrowed during implementation to |
||||
BLOCK-FORM values only (first line whitespace-only): the structural first |
||||
newline and a whitespace-only trailing line are dropped (the user's |
||||
leading-newline ergonomics question - resolved as automatic), the content |
||||
lines' common leading whitespace is the base, first content line unindented |
||||
fully, subsequent lines re-based to 4 spaces with deeper relative indents |
||||
preserved. -unindentedfields fields exempt. |
||||
- Why block-form only: head-form values have an unknowable base - continuations |
||||
uniformly at 6 may be base-4 with the deliberate +2 relative convention (P2) |
||||
or base-6 flush; common-prefix re-basing flattens the former. The idempotence |
||||
test caught exactly this on a file-style def, forcing the narrowing. Block |
||||
form is unambiguous (the first content line carries the full base), and |
||||
head-form untouchedness makes @normalize a proven no-op on conforming |
||||
file-style definitions. |
||||
- Consumer proof: define_docs converted from interim left-margin authoring to |
||||
indented block-form values + @normalize (per the increment 2 decision); |
||||
-unindentedfields declarations dropped; 'i help' and 'i help_chunks' |
||||
verified aligned in punk902z src (including the blank-line separator in the |
||||
combined basehelp+extra block). |
||||
- rendering_constructed_def_indent_characterization stays pinned as the |
||||
deliberate unopted default, its description/header updated to reference |
||||
@normalize as the opt-in remedy (the acceptance's "expectations updated to |
||||
the chosen semantics" - the chosen semantics being opt-in, unopted |
||||
behaviour is contract, not gap). |
||||
- New testsuite normalize.test (5 tests): block-form re-base, block-form |
||||
left-margin, head-form untouched boundary, -unindentedfields exemption |
||||
(structural newline kept byte-exact), idempotence on file-style defs. |
||||
- Verified (tclsh 9.0.3): punk::args suite 186 pass / 1 pre-existing skip / 0 |
||||
fail; punk::ns suite 53/53; full source-tree suite 820 total, 806 pass, 13 |
||||
skip, 1 fail = exec-14.3 only (known pre-existing core-test baseline) - zero |
||||
regressions; make.tcl modules builds clean; 'i help'/'i help_chunks' verified |
||||
aligned in punk902z src. |
||||
|
||||
### 2026-07-12 increment 5: define quoting documentation (punk::args 0.8.1) - acceptance complete |
||||
|
||||
- The container quoting rules pinned by defquoting.test now appear in the |
||||
punk::args::define -help documentation (-help key section): braced values |
||||
fully literal; double-quoted values with Tcl backslash semantics at record |
||||
parse ($ and [] literal, no substitution outside tstr placeholders); the |
||||
backslash-escaped placeholder idiom. Rendered output of the new section |
||||
verified in punk902z src - the meta-escapes display as intended (\n as two |
||||
characters, \Deleted, doubled backslash, literal dollar-brace placeholders). |
||||
- punk::args suite 186 pass / 1 pre-existing skip / 0 fail (tclsh 9.0.3). |
||||
|
||||
Acceptance review at flip (2026-07-12): all criteria satisfied. |
||||
- -& record continuation parses identically to its backslash equivalent |
||||
(recordcontinuation.test byte-equality; additive - existing definitions |
||||
unchanged); collision rules and the {-&} escape documented (increment 3). |
||||
- @cmd -help/-summary honour -unindentedfields; |
||||
rendering_unindentedfields_cmd_help_GAP flipped to aligned (increment 1; |
||||
-summary vacuously honoured - no renderer transforms it, verified). |
||||
- Constructed definitions can request whole-block normalization (@normalize, |
||||
block-form values); rendering_constructed_def_indent_characterization |
||||
updated to the chosen opt-in semantics (unopted behaviour pinned as |
||||
contract); ::punk::helptopic::define_docs dropped its manual |
||||
pre-normalization - first to interim left-margin authoring (increment 2), |
||||
then to indented blocks + @normalize as the consumer proof (increment 4). |
||||
- Quoting rules from defquoting.test appear in the define -help documentation |
||||
(increment 5). |
||||
- Full punk::args suite passes with GAP tests flipped, none weakened: 186 |
||||
pass / 1 pre-existing skip / 0 fail (suite grown from 128 to 187 tests via |
||||
the new recordcontinuation.test and normalize.test); full source-tree suite |
||||
showed zero regressions at increments 3 and 4 (exec-14.3 known baseline |
||||
only). Verified on native tclsh 9.0.3 plus punk902z src for rendered |
||||
output; punk 0.2.5 / punk::args 0.6.1-0.8.1 shipped in project versions |
||||
0.12.4-0.12.8. |
||||
@ -0,0 +1,130 @@
|
||||
# G-071 punk::args value-allocation correctness for optional elements (lseq-class arglists) + parse_status -form |
||||
|
||||
Status: achieved 2026-07-12 |
||||
Scope: src/modules/punk/args-999999.0a1.0.tm (get_dict value allocation, private::get_dict_can_assign_value, parse_status), src/tests/modules/punk/args/testsuites/args/ (new allocation characterization suite), src/modules/punk/args/moduledoc/tclcore-999999.0a1.0.tm (::lseq as the proving consumer) |
||||
Goal: argument lists that are valid for a single form parse correctly when optional standalone values and optional-member clauses are skipped or filled in any documented combination - specifically the lseq range shape (an optional single-word choice value between required values, followed by a trailing optional-member clause), where today the allocator force-feeds a word to the skippable optional and fails ('lseq 0 10 2', '0 10 by 2', '1 5 by 0' all fail in-form while '0 to 10 2' parses) - and parse failures blame the genuinely failing element rather than an unrelated optional; parse_status gains -form so per-form status probing works. |
||||
Acceptance: a new allocation characterization suite drives the lseq range matrix under explicit -form range - '0 10', '0 10 2', '0 10 by 2', '0 to 10 2', '0 .. 10 by 2', '1 5 by 0' parse per the lseq.n grammar (the three currently-failing cases fixed) and '0 10 2 4' still fails - plus reduced fixtures isolating the shape (optional choice value between required values + trailing optional-member clause) independent of the moduledoc; the ::if noise-word cases (mid-clause ?literal(then)?, clause-leading ?literal(else)?) keep passing - no regression to optional clause members generally; a genuinely invalid arglist's error names the failing element (the '..|to' misblame case pinned fixed); punk::args::parse_status accepts -form (single form name/index at minimum, consistent with parse) with its status structure reporting the form used; full punk::args and punk::ns suites pass with no expectations weakened; before/after results for the probe matrix recorded in this file. |
||||
|
||||
## Context |
||||
|
||||
Found 2026-07-12 during G-041 activation prework (user concern: variable-length |
||||
clauses with optional elements). Real-vs-model probing of the tclcore moduledoc |
||||
showed the per-form models are doc-faithful (lseq.n's three synopsis lines map 1:1 |
||||
to the modelled range/start_count/count forms; if/switch/try likewise verified), |
||||
but the range form fails IN-FORM whenever the ../to noise word is absent and a |
||||
step is present - the allocator insists on assigning the word after start to the |
||||
optional '..|to' choice value (whose choices then reject it) instead of skipping |
||||
the optional and letting the trailing "by step" clause (-type |
||||
{?literalprefix(by)? number|expr}) absorb the remainder. The start_count form |
||||
proves the optional-member clause itself can absorb a lone word ('1 count 5 2' |
||||
parses as {by step} {{} 2}) when preceding words are pinned by a literal - the |
||||
failure is the combination with a skippable standalone optional value. |
||||
|
||||
No other lseq form accepts '0 10 2', so G-041's form auto-selection cannot fix |
||||
'i lseq 0 10 2' without this - making this goal a G-041 prerequisite. Probe |
||||
results are recorded in goals/G-041-punkargs-form-matching.md Notes (2026-07-12). |
||||
|
||||
parse_status -form: per-form probing during the same session had to fall back to |
||||
punk::args::parse + catch because parse_status does not accept -form. The |
||||
form-candidacy API sketched in G-041 (and the G-044 partial-arglist consumer) |
||||
wants exactly this surface, so it lands here where the characterization suite |
||||
needs it first. |
||||
|
||||
## Approach |
||||
|
||||
- Characterize first: reduced fixtures pinning current allocation behaviour for |
||||
the failing shape (and the working ::if shapes as regression guards), then fix |
||||
the allocator - likely in the lookahead/backtrack decision where |
||||
get_dict_can_assign_value rejects an optional's candidate word: rejection of an |
||||
optional element should yield the word to subsequent elements/clauses rather |
||||
than failing the parse when a consistent allocation exists. |
||||
- Blame quality: when no consistent allocation exists, prefer reporting the |
||||
element whose constraint actually failed over the first optional tried. |
||||
- The G-041 detail file notes the form-attempt logic will exercise these paths |
||||
heavily; landing this first keeps that goal's failures meaningful. |
||||
|
||||
## Progress |
||||
|
||||
### 2026-07-12 increment 1: characterization suite + retreat-path debug silencing (punk::args 0.8.3) |
||||
|
||||
- New allocation.test (7 tests): reduced lseq-range fixture (start ?sep? end |
||||
?by-step clause? - no moduledoc dependency) with the current mis-allocation |
||||
pinned as GAPs (allocation_range_optskip_bare_step_GAP '1 2 3' -> invalid/sep, |
||||
allocation_range_optskip_by_clause_GAP '1 2 by 3' -> incomplete/end, |
||||
allocation_range_excess_invalid_misblame_GAP '1 2 3 4' -> invalid but blaming |
||||
'sep'), noise-word variants pinned as regression guards (all parse with |
||||
expected receivednames), the if-shape noise-word guards (mid-clause |
||||
?literal(then)?, clause-leading ?literal(else)? - 7 valid + 1 invalid case), |
||||
and parsestatus_form_option_GAP pinning that -form raises today. |
||||
- Fixture probing confirmed the reduced shape reproduces the moduledoc failures |
||||
exactly (same badarg, same status classes), so the fix can iterate against the |
||||
fixtures alone. |
||||
- Silenced four more unconditional debug puts on get_dict's allocation retreat |
||||
paths ((111)/(222)/(333)/(444)) - they fired on stderr during NORMAL |
||||
successful parses at every optional-skip retreat (visible when the fixture |
||||
probes ran; also polluting interactive shell parses of if/lseq-modelled |
||||
commands). Behaviour unchanged - the retreat logic they marked is this goal's |
||||
fix target. |
||||
- Verified (tclsh 9.0.3): full punk::args suite 193 pass / 1 pre-existing skip / |
||||
0 fail. |
||||
|
||||
### 2026-07-12 increment 2: allocation choice screen (punk::args 0.9.0) - acceptance complete |
||||
|
||||
- Root cause: get_dict_can_assign_value blanket-satisfied the member check for any |
||||
argument with choices ("each tp in the clause is just for validating a value |
||||
outside the choice-list when -choicerestricted 0" - applied to restricted sets |
||||
too), so an optional choice value consumed any word whenever arity permitted. |
||||
Arity always permitted here because (a) the tail-reservation scan cannot see the |
||||
by-step clause (its lsearch for literal* misses the optional-wrapped |
||||
?literalprefix(by)?), and (b) tail_needs counts only required tail elements. |
||||
- Fix: a choiceword_match allocation screen (the shared G-040 implementation, so |
||||
allocation acceptance cannot diverge from parse acceptance - exact/alias/prefix/ |
||||
nocase; -choicemultiple words screened per list member within min/max), applied |
||||
ONLY where allocation has an alternative: -optional arguments and further |
||||
occurrences of -multiple arguments. Two refinements forced by the existing |
||||
suites during implementation, both kept as design decisions: |
||||
- REQUIRED arguments are not screened - the word must fill them regardless, and |
||||
screening only masked validation's informative choiceviolation as a |
||||
missingrequired* (caught by parsestatus.test/cmdhelp.test pins). |
||||
- -choicemultiple list-valued words are screened member-wise (caught by |
||||
choices.test choicemultiple pins). |
||||
The screen compares raw words (no ansistrip): an ansi-wrapped choice word on an |
||||
optional argument would be skipped where validation would accept it - accepted |
||||
edge, recorded in the code comment. |
||||
- Correction of an increment-1 mis-premise: parse_status ALREADY accepts -form - |
||||
positioned before the withid/withdef tail per its documented synopsis; the |
||||
probing failure was a trailing -form (argument-order mistake). The GAP was |
||||
replaced by parsestatus_form_option_order pinning correct-order acceptance |
||||
(with the status structure's form key) and trailing-order rejection. The |
||||
acceptance clause "parse_status accepts -form ... reporting the form used" was |
||||
therefore already satisfied; verified with a multiform fixture (form=beta |
||||
reported). The Goal line's "parse_status gains -form" framing was inaccurate - |
||||
no code change was needed or made for it. |
||||
- Before/after (reduced fixture, tclsh 9.0.3): '1 2 3' {0 invalid sep} -> |
||||
{1 valid {start end {by step}}} with clause {{} 3}; '1 2 by 3' |
||||
{0 incomplete end} -> {1 valid ...} with clause {by 3}; '1 2 3 4' |
||||
{0 invalid sep} -> {0 invalid {}} (same excess-values rejection as the |
||||
noise-word twin); prefix semantics preserved ('1 2 b 3' -> clause {by 3}, |
||||
'1 t 2 3' -> sep normalized to 'to'); all noise-word variants and all if-shape |
||||
guards unchanged. |
||||
- lseq moduledoc matrix (punk902z src, Tcl 9.0.2): explicit -form range now |
||||
parses '0 10 2', '0 10 by 2', '1 5 by 0' per lseq.n (previously in-form |
||||
failures); count and start_count forms unchanged; default-form parsing of |
||||
'0 10 2' now valid end-to-end (the original 'i lseq 0 10 2' complaint). |
||||
Remaining real-vs-model divergences all trace to G-041 form selection or the |
||||
-type expr non-validation recorded in G-055's notes - not allocation. |
||||
- Verified (tclsh 9.0.3): punk::args suite 193 pass / 1 pre-existing skip / 0 |
||||
fail (allocation GAPs flipped; the two required-choice-arg pins and the |
||||
choicemultiple pins unweakened); punk::ns suite 53/53; full source-tree suite |
||||
827 total, 813 pass, 13 skip, 1 fail = exec-14.3 only (known pre-existing |
||||
core-test baseline) - zero regressions; make.tcl modules builds clean. |
||||
|
||||
## Notes |
||||
|
||||
- Related: G-041 (this is its prerequisite; probe evidence in its Notes), G-053 |
||||
(occurrence-arity work touches adjacent allocation machinery), G-055 (lseq |
||||
parity pins; TIP 746 note there makes lseq operand types version-conditional - |
||||
this goal's matrix should derive expectations from the live interpreter). |
||||
- Incidental fixes already landed during the probing session: unconditional debug |
||||
puts on the clause type-check path (punk::args 0.8.2), and an earlier companion |
||||
(0.2.3, noted in G-041). |
||||
@ -1,4 +1,4 @@
|
||||
[project] |
||||
name = "punkshell" |
||||
version = "0.12.1" |
||||
version = "0.12.20" |
||||
license = "BSD-2-Clause" |
||||
|
||||
File diff suppressed because it is too large
Load Diff
@ -1,4 +1,8 @@
|
||||
0.2.0 |
||||
0.3.3 |
||||
#First line must be a semantic version number |
||||
#all other lines are ignored. |
||||
#0.3.3 - removed the parse-inert @dynamic tags from ::split, ::array and ::join (user-directed; identified by the 0.3.2 sweep). None has round-2 substitution content, and their ${...} placeholders are all display-field styling/examples - the definitions now resolve as ordinary static definitions with display expansion cached per raw definition (previously @dynamic forced display re-expansion on every render). No bad-@dynamic warnings remain across the module's 462 registered ids; renders and parses verified unchanged for all three. |
||||
#0.3.2 - fixed malformed ::tcl_startOfNextWord definition: its -help was double-quoted but the embedded man-page example contains inner double quotes (set theString "The quick brown fox" / puts "Word start index: ..."), so the value terminated early and the definition failed to resolve at all (punk::args::resolve 'bad optionspecs line' error; 'i tcl_startOfNextWord' broken). The -help is now braced (fully literal per the define quoting rules - inner quotes and brackets safe, text verbatim, example braces balance). Found by a bad-@dynamic sweep across all 462 registered ids (punk::args 0.11.2 work); the same sweep identified the parse-inert @dynamic tags on ::split, ::array and ::join (each now warns once per interp) - left in place pending a decision on their display-refresh semantics (@dynamic still forces display-field re-expansion per render). |
||||
#0.3.1 - authoring-style only (user-directed): the ::after id-shape harvest is consumed via tstr ${$after_id_prefix} placeholders instead of the 0.3.0 build-time %AFTERIDPREFIX% string map - the harvest variable now lives in the argdoc namespace (the defspace registered PUNKARGS definitions resolve placeholders in when an argdoc child exists; see the punk::args::define 'Interpolation and the defspace' help section), which makes plain placeholders work in both the -type parse field (expanded at first resolve) and the -help display fields (expanded at display time). This module showcases the tstr style; string map remains reserved for cases where build-time substitution is genuinely necessary. Behaviour identical to 0.3.0 (resolved -type, form discrimination, parity pins and help renders re-verified). |
||||
#0.3.0 - ::after cancel-id discrimination (user-directed 2026-07-13): the cancelid and info forms' id argument is typed stringstartswith(<prefix>) with the prefix harvested from the RUNNING interpreter at define time (safe create+cancel probe 'after 999999 {}' / 'after cancel $id' - G-054 technique; prefix is after# on 8.6.11 and 9.0.3, hardcoded after#%d in tclTimer.c), substituted into the definition via a build-time %AFTERIDPREFIX% string map (CORRECTED finding: parse-field tstr IS expanded for registered PUNKARGS definitions, but in the argdoc subnamespace when one exists - a variable set in the parent namespace is unresolvable there and the param is left silently literal; build-time substitution sidesteps the defspace subtlety). -typesynopsis id keeps the synopsis rendering as the man page's 'id'. Effect under G-041 form candidacy: 'after cancel <non-id-shaped-word>' resolves cleanly to the cancelscript form matching real semantics (real 'after cancel' with a non-id is a silent script-match no-op), and 'after info <non-id>' is model-rejected where real errors at runtime (parity-true); an id-SHAPED word after cancel remains truthfully ambiguous (cancelid+cancelscript) - real Tcl resolves that junction by id liveness at runtime, which no static type expresses; dead-id over-acceptance on 'after info' recorded as the accepted runtime-liveness boundary. Both ids gain man-page-derived -help text. Parity pins added in tclcoreparity.test (id-shape harvest agreement, cancel discrimination incl the liveness ambiguity, info error-vs-ok parity + accepted dead-id divergence). |
||||
#0.2.0 - G-054: 'string is' class choices (and the generated per-class virtual docids) are harvested from the RUNNING interpreter at define time instead of a hand-maintained list - a deliberately invalid probe of the builtin yields the authoritative class set from its error message (8.6: 21 classes, no dict; unreleased 8.7: +dict +unicode; 9.0: +dict, unicode removed), fixing accept/reject drift such as the doc wrongly accepting 'string is dict' under 8.6. Hand-written man-page descriptions apply only to classes the runtime accepts (generic label for unrecognized future classes); static version notes on dict (not in 8.6) and unicode (unreleased 8.7 only). Parity pinned by tclcoreparity.test with expectations derived from the live interpreter (green on 8.6.13, 8.7a6, 9.0.3) |
||||
|
||||
@ -0,0 +1,162 @@
|
||||
package require tcltest |
||||
|
||||
package require punk::args |
||||
|
||||
#Value-allocation characterization for optional standalone values and |
||||
#optional-member clauses (G-071) - added 2026-07-12 before any allocator changes |
||||
#(tests-first per the punk::args convention). Reduced fixtures isolate the |
||||
#lseq-range shape (start ?sep? end ?by-step clause?) and the if shape (noise-word |
||||
#clauses) from the tclcore moduledoc, so the pins do not depend on moduledoc |
||||
#availability or vintage. |
||||
# |
||||
#History: added with GAP pins encoding the pre-G-071 broken allocation (the |
||||
#allocator blanket-accepted any word for a restricted-choice value at allocation |
||||
#time, so the optional 'sep' noise word greedily consumed the word after start |
||||
#and '1 2 3' / '1 2 by 3' failed in-form with a trailing-choices error). |
||||
#G-071 added a choiceword_match allocation screen for restricted choice sets |
||||
#(punk::args 0.9.0): a non-choice word is no longer allocatable to 'sep', so it |
||||
#yields onward to end and the by-step clause - the GAPs below flipped to the |
||||
#correct outcomes the same day they were added. |
||||
|
||||
namespace eval ::testspace { |
||||
namespace import ::tcltest::* |
||||
variable common { |
||||
set result "" |
||||
punk::args::define { |
||||
@id -id ::testspace::alloc_range |
||||
@values -min 2 -max 5 |
||||
start -type int |
||||
sep -type string -choices {.. to} -optional 1 |
||||
end -type int |
||||
"by step" -type {?literalprefix(by)? int} -optional 1 |
||||
} |
||||
punk::args::define { |
||||
@id -id ::testspace::alloc_if |
||||
@values -min 2 -max -1 |
||||
expr1 -type int -optional 0 |
||||
then -type literal(then) -optional 1 |
||||
body1 -type int -optional 0 |
||||
"elseif_clause" -type {literal(elseif) int ?literal(then)? int} -optional 1 -multiple 1 |
||||
"else_clause" -type {?literal(else)? int} -optional 1 -multiple 0 |
||||
} |
||||
} |
||||
variable cleanup { |
||||
punk::args::undefine ::testspace::alloc_range 1 |
||||
punk::args::undefine ::testspace::alloc_if 1 |
||||
} |
||||
#ok/status plus badarg (failed) or receivednames (ok) - compact per-case signature |
||||
proc pstat {arglist id} { |
||||
set st [punk::args::parse_status $arglist withid $id] |
||||
if {[dict get $st ok]} { |
||||
return [list 1 [dict get $st status] [dict get $st receivednames]] |
||||
} |
||||
return [list 0 [dict get $st status] [dict get $st badarg]] |
||||
} |
||||
|
||||
#added 2026-07-12 (agent, G-071) - regression guards: allocation with the |
||||
#optional noise word present (or no step) works today and must keep working |
||||
test allocation_range_noiseword_variants {lseq-range shape: variants with the sep noise word present (or only start end) allocate correctly}\ |
||||
-setup $common -body { |
||||
lappend result [pstat {1 2} ::testspace::alloc_range] |
||||
lappend result [pstat {1 to 2} ::testspace::alloc_range] |
||||
lappend result [pstat {1 to 2 3} ::testspace::alloc_range] |
||||
lappend result [pstat {1 .. 2 by 3} ::testspace::alloc_range] |
||||
lappend result [pstat {1 to 2 by 3} ::testspace::alloc_range] |
||||
}\ |
||||
-cleanup $cleanup\ |
||||
-result [list\ |
||||
{1 valid {start end}}\ |
||||
{1 valid {start sep end}}\ |
||||
{1 valid {start sep end {by step}}}\ |
||||
{1 valid {start sep end {by step}}}\ |
||||
{1 valid {start sep end {by step}}}\ |
||||
] |
||||
|
||||
#added 2026-07-12 (agent, G-071) - was allocation_range_optskip_bare_step_GAP |
||||
#(pinned {0 invalid sep}); flipped by the G-071 allocation choice screen |
||||
test allocation_range_optskip_bare_step {sep absent + bare step ('1 2 3'): non-choice word yields past the optional 'sep' to end + clause}\ |
||||
-setup $common -body { |
||||
lappend result [pstat {1 2 3} ::testspace::alloc_range] |
||||
#the clause's optional literalprefix member is omitted (empty) |
||||
lappend result [dict get [dict get [punk::args::parse {1 2 3} withid ::testspace::alloc_range] values] {by step}] |
||||
}\ |
||||
-cleanup $cleanup\ |
||||
-result [list {1 valid {start end {by step}}} {{} 3}] |
||||
|
||||
#added 2026-07-12 (agent, G-071) - was allocation_range_optskip_by_clause_GAP |
||||
#(pinned {0 incomplete end}); flipped by the G-071 allocation choice screen |
||||
test allocation_range_optskip_by_clause {sep absent + by-clause ('1 2 by 3'): allocation yields correctly and the clause takes both members}\ |
||||
-setup $common -body { |
||||
lappend result [pstat {1 2 by 3} ::testspace::alloc_range] |
||||
lappend result [dict get [dict get [punk::args::parse {1 2 by 3} withid ::testspace::alloc_range] values] {by step}] |
||||
#prefix semantics survive the allocation screen: 'b' prefixes the clause |
||||
#literal, 't' prefixes the sep choice (normalized to 'to') |
||||
lappend result [dict get [dict get [punk::args::parse {1 2 b 3} withid ::testspace::alloc_range] values] {by step}] |
||||
lappend result [dict get [dict get [punk::args::parse {1 t 2 3} withid ::testspace::alloc_range] values] sep] |
||||
}\ |
||||
-cleanup $cleanup\ |
||||
-result [list {1 valid {start end {by step}}} {by 3} {by 3} to] |
||||
|
||||
#added 2026-07-12 (agent, G-071) - was allocation_range_excess_invalid_misblame_GAP |
||||
#(pinned {0 invalid sep}); the arglist stays invalid (per lseq.n no form accepts |
||||
#it) but the blame no longer lands on the unrelated optional 'sep' - it is now |
||||
#the same excess-values rejection as the noise-word twin '1 to 2 3 4' |
||||
test allocation_range_excess_invalid {genuinely invalid '1 2 3 4' is rejected as excess values without blaming 'sep'}\ |
||||
-setup $common -body { |
||||
pstat {1 2 3 4} ::testspace::alloc_range |
||||
}\ |
||||
-cleanup $cleanup\ |
||||
-result {0 invalid {}} |
||||
|
||||
#added 2026-07-12 (agent, G-071) - guard: with the noise word present the |
||||
#equivalent excess arglist is rejected without touching 'sep' |
||||
test allocation_range_excess_with_noiseword_invalid {lseq-range shape: '1 to 2 3 4' rejected (excess values)}\ |
||||
-setup $common -body { |
||||
set st [punk::args::parse_status {1 to 2 3 4} withid ::testspace::alloc_range] |
||||
list [dict get $st ok] [dict get $st status] |
||||
}\ |
||||
-cleanup $cleanup\ |
||||
-result {0 invalid} |
||||
|
||||
#added 2026-07-12 (agent, G-071) - regression guards: the ::if noise-word |
||||
#shapes (mid-clause ?literal(then)?, clause-leading ?literal(else)?) allocate |
||||
#correctly today and must not regress with the allocator fix |
||||
test allocation_if_noiseword_guards {if shape: mid-clause and clause-leading optional literals allocate correctly}\ |
||||
-setup $common -body { |
||||
foreach c { |
||||
{1 2} |
||||
{1 then 2} |
||||
{1 2 else 3} |
||||
{1 2 3} |
||||
{1 2 elseif 3 4} |
||||
{1 2 elseif 3 then 4 else 5} |
||||
{1 then 2 elseif 3 then 4 elseif 5 6 else 7} |
||||
} { |
||||
set st [punk::args::parse_status $c withid ::testspace::alloc_if] |
||||
lappend result [list [dict get $st ok] [dict get $st status]] |
||||
} |
||||
#incomplete elseif clause is rejected |
||||
set st [punk::args::parse_status {1 2 elseif 3} withid ::testspace::alloc_if] |
||||
lappend result [list [dict get $st ok] [dict get $st status]] |
||||
}\ |
||||
-cleanup $cleanup\ |
||||
-result [list {1 valid} {1 valid} {1 valid} {1 valid} {1 valid} {1 valid} {1 valid} {0 invalid}] |
||||
|
||||
#added 2026-07-12 (agent, G-071) - characterization correcting an initial |
||||
#mis-premise: parse_status DOES accept -form, positioned per its documented |
||||
#synopsis (options BEFORE the withid/withdef tail); a trailing -form is |
||||
#rejected like any misplaced option |
||||
test parsestatus_form_option_order {parse_status -form works before the withid tail and is rejected trailing}\ |
||||
-setup $common -body { |
||||
set st [punk::args::parse_status {1 to 2} -form 0 withid ::testspace::alloc_range] |
||||
lappend result [list [dict get $st ok] [dict get $st status]] |
||||
#the status structure reports the form used |
||||
lappend result [dict get $st form] |
||||
lappend result [catch {punk::args::parse_status {1 to 2} withid ::testspace::alloc_range -form 0}] |
||||
}\ |
||||
-cleanup $cleanup\ |
||||
-result [list {1 valid} _default 1] |
||||
|
||||
cleanupTests |
||||
} |
||||
namespace delete ::testspace |
||||
@ -0,0 +1,96 @@
|
||||
package require tcltest |
||||
|
||||
package require punk::args |
||||
|
||||
#@dynamic definition resolution behaviour - added 2026-07-13 (agent, user-reported |
||||
#repeated bad-@dynamic warnings from 'i join'). |
||||
#A definition marked @dynamic whose round-1 tstr output contains no round-2 ${...} |
||||
#parameters is a "bad @dynamic tag": the tag buys nothing (display-field ${...} is |
||||
#deferred regardless - see the define help's interpolation section). resolve warns - |
||||
#and since 2026-07-13 caches the round-1 output as a zero-param unresolved entry, so |
||||
#the warning emits ONCE per definition and later resolves skip the re-mask/re-tstr of |
||||
#round 1 (the same round-1 freezing legitimately dynamic definitions get). |
||||
#Legitimately dynamic definitions are unaffected: round-2 parameters re-substitute on |
||||
#every resolve. |
||||
|
||||
namespace eval ::testspace { |
||||
namespace import ::tcltest::* |
||||
variable common { |
||||
set result "" |
||||
} |
||||
|
||||
#capture stderr written during script (chan push transform; popped in all paths) |
||||
proc capture_stderr {script} { |
||||
variable caught "" |
||||
chan push stderr {apply {{cmd chan args} { |
||||
switch -- $cmd { |
||||
initialize {return {initialize finalize write}} |
||||
finalize {return} |
||||
write {append ::testspace::caught [lindex $args 0]; return ""} |
||||
} |
||||
}}} |
||||
set code [catch {uplevel 1 $script} r ropts] |
||||
chan pop stderr |
||||
if {$code} { |
||||
return -options $ropts $r |
||||
} |
||||
return $caught |
||||
} |
||||
|
||||
test dynamic_bad_tag_warns_once {a @dynamic definition with no round-2 parameters warns once and is served from the round-1 cache thereafter}\ |
||||
-setup $common -body { |
||||
punk::args::define { |
||||
@dynamic |
||||
@id -id ::testspace::baddyn |
||||
@cmd -name testspace::baddyn -help "display-only placeholder ${$display_only_content}" |
||||
@values -min 1 -max 1 |
||||
v -type string |
||||
} |
||||
set errout [capture_stderr { |
||||
punk::args::get_spec ::testspace::baddyn |
||||
punk::args::get_spec ::testspace::baddyn |
||||
punk::args::parse {hello} withid ::testspace::baddyn |
||||
punk::args::parse_status {hello} withid ::testspace::baddyn |
||||
}] |
||||
lappend result [regexp -all {bad @dynamic tag for id:::testspace::baddyn} $errout] |
||||
#the cached round-1 output still parses correctly |
||||
set argd [punk::args::parse {world} withid ::testspace::baddyn] |
||||
lappend result [dict get $argd values v] |
||||
}\ |
||||
-cleanup { |
||||
punk::args::undefine ::testspace::baddyn 1 |
||||
}\ |
||||
-result [list 1 world] |
||||
|
||||
test dynamic_legit_round2_stays_live {a legitimately dynamic definition re-substitutes round-2 parameters on every resolve, with no bad-tag warning}\ |
||||
-setup $common -body { |
||||
variable dynchoices {a b} |
||||
proc ::testspace::choices_now {} { |
||||
variable dynchoices |
||||
return $dynchoices |
||||
} |
||||
namespace eval ::testspace::dynfix { |
||||
set DYN_CHOICES {${[::testspace::choices_now]}} |
||||
punk::args::define { |
||||
@dynamic |
||||
@id -id ::testspace::gooddyn |
||||
@cmd -name testspace::gooddyn -help "legit dynamic" |
||||
@values -min 1 -max 1 |
||||
v -choices {${$DYN_CHOICES}} |
||||
} |
||||
} |
||||
set errout [capture_stderr { |
||||
lappend result [dict get [punk::args::get_spec ::testspace::gooddyn] FORMS _default ARG_INFO v -choices] |
||||
set ::testspace::dynchoices {c d} |
||||
lappend result [dict get [punk::args::get_spec ::testspace::gooddyn] FORMS _default ARG_INFO v -choices] |
||||
}] |
||||
lappend result [regexp -all {bad @dynamic tag} $errout] |
||||
}\ |
||||
-cleanup { |
||||
namespace delete ::testspace::dynfix |
||||
rename ::testspace::choices_now {} |
||||
punk::args::undefine ::testspace::gooddyn 1 |
||||
}\ |
||||
-result [list {a b} {c d} 0] |
||||
} |
||||
tcltest::cleanupTests ;#needed to produce test summary line. |
||||
@ -0,0 +1,192 @@
|
||||
package require tcltest |
||||
|
||||
package require punk::args |
||||
package require punk::ansi |
||||
|
||||
#@normalize directive (G-045): opt-in indent normalization of BLOCK-FORM multi-line |
||||
#field values for constructed (string-built) definitions. Semantics (user-confirmed |
||||
#re-base 2026-07-12; narrowed to block form during implementation): a value whose |
||||
#first line is whitespace-only is a block - the structural first newline and a |
||||
#whitespace-only trailing line are dropped, the content lines' common leading |
||||
#whitespace is the block's base indent, the first content line is unindented fully |
||||
#and subsequent lines are re-based to the 4-space convention (deeper relative |
||||
#indents preserved). Head-form values are NEVER altered: their base is ambiguous |
||||
#(uniform continuation indent may be the deliberate +2 relative convention over |
||||
#base 4, or a deeper flush base) - which also makes @normalize a no-op on |
||||
#conforming file-style definitions. Fields in a record's -unindentedfields are |
||||
#exempt. Without @normalize, constructed-def behaviour is unchanged - see the P4 |
||||
#characterization in rendering.test. |
||||
|
||||
namespace eval ::testspace { |
||||
namespace import ::tcltest::* |
||||
variable common { |
||||
set result "" |
||||
} |
||||
|
||||
proc render_table {id} { |
||||
return [punk::ansi::ansistrip [punk::args::arg_error "" [punk::args::get_spec $id] -aserror 0]] |
||||
} |
||||
#column (0-based) at which a unique marker string appears in the rendered text, -1 if absent |
||||
proc markercol {rendered marker} { |
||||
foreach ln [split $rendered \n] { |
||||
set ix [string first $marker $ln] |
||||
if {$ix >= 0} { |
||||
return $ix |
||||
} |
||||
} |
||||
return -1 |
||||
} |
||||
|
||||
#added 2026-07-12 (agent, G-045) - the head-form boundary: base indent is |
||||
#ambiguous for head-form values so @normalize leaves them alone (deep indents |
||||
#leak exactly as in the unopted P4 characterization: display strips only 4) |
||||
test normalize_headform_untouched {@normalize: head-form values (content on the first line) are not altered - deep continuation indents render as authored}\ |
||||
-setup $common -body { |
||||
set help "NFIRST line\n NFLUSH line\n NPLUS2 line" |
||||
set def "" |
||||
append def "@normalize" \n |
||||
append def "@id -id ::testspace::nz_head" \n |
||||
append def "@cmd -name testspace::nz_head -summary \"Head.\" -help \"$help\"" \n |
||||
append def "@values -min 0 -max 0" |
||||
punk::args::define $def |
||||
set r [render_table ::testspace::nz_head] |
||||
set n0 [markercol $r NFIRST] |
||||
lappend result [expr {[markercol $r NFLUSH] - $n0}] |
||||
lappend result [expr {[markercol $r NPLUS2] - $n0}] |
||||
}\ |
||||
-cleanup { |
||||
punk::args::undefine ::testspace::nz_head 1 |
||||
}\ |
||||
-result [list 12 14] |
||||
|
||||
#added 2026-07-12 (agent, G-045) - the block-form ergonomics: a braced value |
||||
#authored as a clean indented block (leading newline, closing-brace line) needs |
||||
#no manual string trim |
||||
test normalize_blockform_leading_newline {@normalize: block-form value (whitespace-only first line) drops the structural newline and renders flush}\ |
||||
-setup $common -body { |
||||
set help { |
||||
BFIRST line |
||||
BFLUSH line |
||||
BPLUS2 line |
||||
} |
||||
set def "" |
||||
append def "@normalize" \n |
||||
append def "@id -id ::testspace::nz_block" \n |
||||
append def "@cmd -name testspace::nz_block -summary \"Block.\" -help \"$help\"" \n |
||||
append def "@values -min 0 -max 0" |
||||
punk::args::define $def |
||||
set r [render_table ::testspace::nz_block] |
||||
set b0 [markercol $r BFIRST] |
||||
lappend result [expr {[markercol $r BFLUSH] - $b0}] |
||||
lappend result [expr {[markercol $r BPLUS2] - $b0}] |
||||
#the structural leading newline is gone: BFIRST appears on the same |
||||
#rendered line as the Description: label |
||||
set desc_line "" |
||||
foreach ln [split $r \n] { |
||||
if {[string first "Description:" $ln] >= 0} { |
||||
set desc_line $ln |
||||
break |
||||
} |
||||
} |
||||
lappend result [expr {[string first "BFIRST" $desc_line] >= 0}] |
||||
}\ |
||||
-cleanup { |
||||
punk::args::undefine ::testspace::nz_block 1 |
||||
}\ |
||||
-result [list 0 2 1] |
||||
|
||||
#added 2026-07-12 (agent, G-045) - block form with zero base indent: left-margin |
||||
#content behind a structural leading newline re-bases to the convention (no |
||||
#-unindentedfields declaration needed) |
||||
test normalize_blockform_leftmargin_gains_base {@normalize: a block-form value with left-margin content is re-based and renders flush}\ |
||||
-setup $common -body { |
||||
set help "\nLFIRST line\nLFLUSH line" |
||||
set def "" |
||||
append def "@normalize" \n |
||||
append def "@id -id ::testspace::nz_left" \n |
||||
append def "@cmd -name testspace::nz_left -summary \"Left.\" -help \"$help\"" \n |
||||
append def "@values -min 0 -max 0" |
||||
punk::args::define $def |
||||
set r [render_table ::testspace::nz_left] |
||||
lappend result [expr {[markercol $r LFLUSH] - [markercol $r LFIRST]}] |
||||
}\ |
||||
-cleanup { |
||||
punk::args::undefine ::testspace::nz_left 1 |
||||
}\ |
||||
-result [list 0] |
||||
|
||||
#added 2026-07-12 (agent, G-045) - exemption: an -unindentedfields field keeps |
||||
#its block-form value byte-exact (structural newline preserved, no re-base, no |
||||
#display transform) where a non-exempt block would lose the leading blank line |
||||
test normalize_unindentedfields_exempt {@normalize: fields in a record's -unindentedfields are exempt - block-form value kept literally}\ |
||||
-setup $common -body { |
||||
set help "\n EFIRST line\n ESIX line" |
||||
set def "" |
||||
append def "@normalize" \n |
||||
append def "@id -id ::testspace::nz_exempt" \n |
||||
append def "@cmd -name testspace::nz_exempt -summary \"Exempt.\" -unindentedfields {-help} -help \"$help\"" \n |
||||
append def "@values -min 0 -max 0" |
||||
punk::args::define $def |
||||
set r [render_table ::testspace::nz_exempt] |
||||
#leading structural newline preserved: EFIRST is NOT on the Description: line |
||||
set desc_line "" |
||||
foreach ln [split $r \n] { |
||||
if {[string first "Description:" $ln] >= 0} { |
||||
set desc_line $ln |
||||
break |
||||
} |
||||
} |
||||
lappend result [expr {[string first "EFIRST" $desc_line] < 0}] |
||||
#6-space indents kept literally - both lines at the same column |
||||
lappend result [expr {[markercol $r ESIX] - [markercol $r EFIRST]}] |
||||
}\ |
||||
-cleanup { |
||||
punk::args::undefine ::testspace::nz_exempt 1 |
||||
}\ |
||||
-result [list 1 0] |
||||
|
||||
#added 2026-07-12 (agent, G-045) - idempotence: @normalize on a conforming |
||||
#file-style braced definition changes nothing |
||||
test normalize_idempotent_on_filestyle {@normalize: a braced file-style definition renders identically with and without the directive}\ |
||||
-setup $common -body { |
||||
punk::args::define { |
||||
@id -id ::testspace::nz_fplain |
||||
@cmd -name testspace::nz_fstyle -summary\ |
||||
"Filestyle."\ |
||||
-help\ |
||||
"FFIRST line. |
||||
FFLUSH line. |
||||
FPLUS2 line." |
||||
@values -min 1 -max 1 |
||||
v1 -type string -help\ |
||||
"val help line1 |
||||
val help line2" |
||||
} |
||||
punk::args::define { |
||||
@normalize |
||||
@id -id ::testspace::nz_fnorm2 |
||||
@cmd -name testspace::nz_fstyle -summary\ |
||||
"Filestyle."\ |
||||
-help\ |
||||
"FFIRST line. |
||||
FFLUSH line. |
||||
FPLUS2 line." |
||||
@values -min 1 -max 1 |
||||
v1 -type string -help\ |
||||
"val help line1 |
||||
val help line2" |
||||
} |
||||
set r_plain [render_table ::testspace::nz_fplain] |
||||
set r_norm [render_table ::testspace::nz_fnorm2] |
||||
#ids deliberately same length (nz_fplain/nz_fnorm2) so table geometry matches |
||||
lappend result [expr {[string map {nz_fnorm2 nz_fplain} $r_norm] eq $r_plain}] |
||||
}\ |
||||
-cleanup { |
||||
punk::args::undefine ::testspace::nz_fplain 1 |
||||
punk::args::undefine ::testspace::nz_fnorm2 1 |
||||
}\ |
||||
-result [list 1] |
||||
|
||||
cleanupTests |
||||
} |
||||
namespace delete ::testspace |
||||
@ -0,0 +1,182 @@
|
||||
package require tcltest |
||||
|
||||
package require punk::args |
||||
package require punk::ansi |
||||
|
||||
#Record-continuation token -& (G-045): an unquoted trailing -& element continues a |
||||
#definition record on the next line, rewritten internally to a backslash |
||||
#line-continuation so the assembled record is byte-identical to its |
||||
#backslash-continued equivalent. Key properties pinned here: |
||||
# - equivalence: a -& definition parses AND renders identically to its |
||||
# backslash-continuation twin (continuation is additive - backslash authoring |
||||
# unchanged) |
||||
# - the motivating case: constructed (string-built) definitions can use -& where |
||||
# a backslash-newline would be consumed by the building code's own quoting |
||||
# - collision/escape rules: a braced trailing {-&} is data; -& mid-line is data; |
||||
# a word merely ending in the characters -& is data; -& on a line inside a |
||||
# still-open braced value is data |
||||
|
||||
namespace eval ::testspace { |
||||
namespace import ::tcltest::* |
||||
variable common { |
||||
set result "" |
||||
} |
||||
|
||||
proc render_table {id} { |
||||
return [punk::ansi::ansistrip [punk::args::arg_error "" [punk::args::get_spec $id] -aserror 0]] |
||||
} |
||||
|
||||
#added 2026-07-12 (agent, G-045) |
||||
#the two ids are deliberately the same length: the id appears in the rendered |
||||
#synopsis, so differing lengths would change table geometry and defeat the |
||||
#render-equality comparison after the id string map |
||||
test recordcontinuation_amp_equivalent_to_backslash {a definition using trailing -& parses and renders identically to its backslash-continuation equivalent}\ |
||||
-setup $common -body { |
||||
punk::args::define { |
||||
@id -id ::testspace::rc_bsl |
||||
@cmd -name testspace::rc_fixture -summary\ |
||||
"Fixture."\ |
||||
-help\ |
||||
"CFIRST line. |
||||
CFLUSH line." |
||||
@opts |
||||
-o1 -type string -default od1 -help\ |
||||
"opt help line1 |
||||
opt help line2" |
||||
@values -min 1 -max 1 |
||||
v1 -type string -help\ |
||||
"val help." |
||||
} |
||||
punk::args::define { |
||||
@id -id ::testspace::rc_amp |
||||
@cmd -name testspace::rc_fixture -summary -& |
||||
"Fixture." -& |
||||
-help -& |
||||
"CFIRST line. |
||||
CFLUSH line." |
||||
@opts |
||||
-o1 -type string -default od1 -help -& |
||||
"opt help line1 |
||||
opt help line2" |
||||
@values -min 1 -max 1 |
||||
v1 -type string -help -& |
||||
"val help." |
||||
} |
||||
set r_b [punk::args::parse {-o1 X VAL1} withid ::testspace::rc_bsl] |
||||
set r_a [punk::args::parse {-o1 X VAL1} withid ::testspace::rc_amp] |
||||
lappend result [expr {[string map {rc_amp rc_bsl} $r_a] eq $r_b}] |
||||
set t_b [render_table ::testspace::rc_bsl] |
||||
set t_a [render_table ::testspace::rc_amp] |
||||
lappend result [expr {[string map {rc_amp rc_bsl} $t_a] eq $t_b}] |
||||
}\ |
||||
-cleanup { |
||||
punk::args::undefine ::testspace::rc_bsl 1 |
||||
punk::args::undefine ::testspace::rc_amp 1 |
||||
}\ |
||||
-result [list 1 1] |
||||
|
||||
#added 2026-07-12 (agent, G-045) - the motivating case: string-built definitions |
||||
#cannot use backslash-newline (consumed by the builder's own quoting) but can use -& |
||||
test recordcontinuation_amp_constructed_definition {a constructed (string-built) definition using -& chains record lines, including into a multi-line quoted value}\ |
||||
-setup $common -body { |
||||
set def "" |
||||
append def "@id -id ::testspace::rc_chain" \n |
||||
append def "@cmd -name testspace::rc_chain -summary \"Chain.\" -help \"chain help\"" \n |
||||
append def "@opts" \n |
||||
append def "-o1 -type string -&" \n |
||||
append def "-default HDEF -& " \n |
||||
append def "-help \"h line1" \n |
||||
append def "h line2\"" \n |
||||
append def "@values -min 0 -max 0" |
||||
punk::args::define $def |
||||
set argd [punk::args::parse {} withid ::testspace::rc_chain] |
||||
lappend result [dict get [dict get $argd opts] -o1] |
||||
set t [render_table ::testspace::rc_chain] |
||||
lappend result [expr {[string first "h line1" $t] >= 0}] |
||||
lappend result [expr {[string first "h line2" $t] >= 0}] |
||||
}\ |
||||
-cleanup { |
||||
punk::args::undefine ::testspace::rc_chain 1 |
||||
}\ |
||||
-result [list HDEF 1 1] |
||||
|
||||
#added 2026-07-12 (agent, G-045) |
||||
test recordcontinuation_amp_braced_escape_is_data {a braced trailing {-&} is a literal value - the record ends and the next line is a separate record}\ |
||||
-setup $common -body { |
||||
punk::args::define { |
||||
@id -id ::testspace::rc_braced |
||||
@cmd -name testspace::rc_braced -summary "Braced." -help "braced help" |
||||
@opts |
||||
-o1 -type string -default {-&} |
||||
-o2 -type string -default d2 |
||||
@values -min 0 -max 0 |
||||
} |
||||
set argd [punk::args::parse {} withid ::testspace::rc_braced] |
||||
lappend result [dict get [dict get $argd opts] -o1] |
||||
lappend result [dict get [dict get $argd opts] -o2] |
||||
}\ |
||||
-cleanup { |
||||
punk::args::undefine ::testspace::rc_braced 1 |
||||
}\ |
||||
-result [list -& d2] |
||||
|
||||
#added 2026-07-12 (agent, G-045) |
||||
test recordcontinuation_amp_midline_is_data {an unquoted -& that is not the last element on the line is an ordinary value}\ |
||||
-setup $common -body { |
||||
punk::args::define { |
||||
@id -id ::testspace::rc_midline |
||||
@cmd -name testspace::rc_midline -summary "Midline." -help "midline help" |
||||
@opts |
||||
-o1 -default -& -type string |
||||
@values -min 0 -max 0 |
||||
} |
||||
set argd [punk::args::parse {} withid ::testspace::rc_midline] |
||||
lappend result [dict get [dict get $argd opts] -o1] |
||||
}\ |
||||
-cleanup { |
||||
punk::args::undefine ::testspace::rc_midline 1 |
||||
}\ |
||||
-result [list -&] |
||||
|
||||
#added 2026-07-12 (agent, G-045) |
||||
test recordcontinuation_amp_wordend_is_data {a word merely ending in the characters -& is not a continuation token}\ |
||||
-setup $common -body { |
||||
punk::args::define { |
||||
@id -id ::testspace::rc_wordend |
||||
@cmd -name testspace::rc_wordend -summary "Wordend." -help "wordend help" |
||||
@opts |
||||
-o1 -type string -default abc-& |
||||
-o2 -type string -default d2 |
||||
@values -min 0 -max 0 |
||||
} |
||||
set argd [punk::args::parse {} withid ::testspace::rc_wordend] |
||||
lappend result [dict get [dict get $argd opts] -o1] |
||||
lappend result [dict get [dict get $argd opts] -o2] |
||||
}\ |
||||
-cleanup { |
||||
punk::args::undefine ::testspace::rc_wordend 1 |
||||
}\ |
||||
-result [list abc-& d2] |
||||
|
||||
#added 2026-07-12 (agent, G-045) |
||||
test recordcontinuation_amp_inside_braces_is_data {a line ending in -& inside a still-open braced value stays literal data}\ |
||||
-setup $common -body { |
||||
punk::args::define { |
||||
@id -id ::testspace::rc_inbrace |
||||
@cmd -name testspace::rc_inbrace -summary "Inbrace." -help "inbrace help" |
||||
@values -min 1 -max 1 |
||||
v1 -type string -help {AFIRST -& |
||||
ASECOND line} |
||||
} |
||||
set t [render_table ::testspace::rc_inbrace] |
||||
lappend result [expr {[string first "AFIRST -&" $t] >= 0}] |
||||
lappend result [expr {[string first "ASECOND line" $t] >= 0}] |
||||
}\ |
||||
-cleanup { |
||||
punk::args::undefine ::testspace::rc_inbrace 1 |
||||
}\ |
||||
-result [list 1 1] |
||||
|
||||
cleanupTests |
||||
} |
||||
namespace delete ::testspace |
||||
Loading…
Reference in new issue