Browse Source
Both user-approved drafts arising from the G-045 tclparser assessment: - G-069: dev-time lint cross-checking punk::args record-splitter boundaries against tclparser parse command where the binary is available - dialect-aware comparison (-& pre-join, ANSI strip), capability-gated, zero new runtime dependencies for punk::args. - G-070: pure-Tcl implementation of the tclparser parse-command API scoped to what punk::lib consumes, parity-verified against the C library over a probe corpus; replaces the tclparser_tcl error stub with prefer-C/fallback-pure dispatch. Reference source to be user-provided under TEMP_REFERENCE with recorded provenance. Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.commaster
3 changed files with 114 additions and 0 deletions
@ -0,0 +1,50 @@ |
|||||||
|
# G-069 Dev-time lint: cross-check punk::args record splitting against tclparser where the binary is available |
||||||
|
|
||||||
|
Status: proposed |
||||||
|
Scope: lint surface (location settled in the work: punk::args dev helper, dev commandset, or scriptlib/developer script), src/modules/punk/args-999999.0a1.0.tm (split_definition_records as consumed, no new dependency), src/tests/modules/punk/args/ (capability-gated suite) |
||||||
|
Goal: a dev-time lint cross-checks the record boundaries produced by punk::args' definition record splitter (private::split_definition_records) against the tclparser C library's 'parse command' tokenization of the same (ANSI-stripped, dialect-adjusted) text, so drift between the definition dialect and real Tcl parsing rules is caught as the dialect grows (-&, @normalize, future continuation/normalization directives) - without punk::args itself gaining any dependency on the tclparser binary. |
||||||
|
Acceptance: a documented lint command (location decided in the work) accepts definition text and/or registered definition ids and reports record-boundary divergence between split_definition_records and tclparser 'parse command'; the comparison accounts for documented dialect features rather than flagging them (at minimum: ANSI escapes stripped before parse, -& record continuation pre-joined or classified as expected divergence); the lint is capability-gated - a clean, explicit skip/notice when the tclparser package is unavailable, never an error - and `package require punk::args` continues to succeed with no tclparser present; a run across all definitions registered in a punk9win kit shell (where the tclparser binary ships) yields zero unexplained divergences, with any real divergences found either fixed or recorded in this file; exercised by a capability-gated test or a documented manual run recorded here. |
||||||
|
|
||||||
|
## Context |
||||||
|
|
||||||
|
While implementing the -& record-continuation token (G-045, archived), tclparser's |
||||||
|
'parse' command was considered and rejected as the splitter's parsing substrate: the |
||||||
|
definition dialect legitimately deviates from Tcl (unbalanced-bracket ANSI escapes as |
||||||
|
data inside quoted values, line-based base-indent semantics, the -& token), and a |
||||||
|
binary dependency in punk::args - the module that must load everywhere, including |
||||||
|
bootsupport contexts and plain tclsh - inverts the G-004 direction. The full analysis |
||||||
|
is recorded in goals/archive/G-045-punkargs-authoring-ergonomics.md (Progress, |
||||||
|
increment 3) and in the punk::args 0.7.0 changelog. |
||||||
|
|
||||||
|
The rejected option still has value as a diagnostic: records are shaped like Tcl |
||||||
|
commands (newline-delimited, quote/brace/backslash continuation), so where the binary |
||||||
|
is available, parse-command boundaries are an independent oracle for the splitter. |
||||||
|
As the dialect grows, an automated cross-check catches unintended divergence early - |
||||||
|
the splitter's own testsuites (recordcontinuation.test, normalize.test, defquoting.test, |
||||||
|
rendering.test) pin known behaviour but cannot flag novel drift against Tcl semantics. |
||||||
|
|
||||||
|
## Approach |
||||||
|
|
||||||
|
- Comparison pipeline: take the definition text as the splitter receives it, produce |
||||||
|
splitter records; separately ansistrip the text, pre-join -& continuations (or |
||||||
|
classify them), then iterate tclparser 'parse command' to produce boundary ranges; |
||||||
|
compare record counts and boundary line positions, reporting divergences with |
||||||
|
context. |
||||||
|
- Corpus: registered definitions via punk::args introspection (update_definitions + |
||||||
|
raw defs), plus ad-hoc text input for testing definition snippets during dialect |
||||||
|
work. |
||||||
|
- Home: this is developer tooling, not runtime behaviour - candidates are a |
||||||
|
punk::args dev/diagnostic namespace proc (loaded lazily), a dev commandset command, |
||||||
|
or a scriptlib/developer script following goals_lint.tcl precedent. Decide in the |
||||||
|
work; the constraint is only that punk::args' runtime footprint is unchanged. |
||||||
|
- The tclparser binary currently ships only in the punk9win kit (lib_tcl9); the lint |
||||||
|
is expected to run there (or any environment with the package installed). If G-070 |
||||||
|
(pure-Tcl tclparser) lands, the gate widens automatically if the lint requires |
||||||
|
'parser' via ordinary package resolution. |
||||||
|
|
||||||
|
## Notes |
||||||
|
|
||||||
|
- Related: G-045 (archived - origin of the rejected-then-repurposed idea and the |
||||||
|
dialect features the comparison must understand), G-070 (pure-Tcl tclparser would |
||||||
|
let the lint run without the binary), G-004 (why the runtime must stay |
||||||
|
dependency-free). |
||||||
@ -0,0 +1,56 @@ |
|||||||
|
# G-070 Pure-Tcl tclparser: parse-command API fallback with behavioural parity against the C library |
||||||
|
|
||||||
|
Status: proposed |
||||||
|
Scope: src/modules/punk/lib-999999.0a1.0.tm (tclparser_tcl stub + dispatch; new module if size warrants - decided in the work), src/tests/modules/punk/lib/ (parity + fallback suites), TEMP_REFERENCE/ (tclparser reference source, user-provided, read-only) |
||||||
|
Goal: punk::lib's script-analysis machinery (tclword_to_scriptlist and the dependent analysis paths) runs on runtimes without the tclparser C binary, via a pure-Tcl implementation of the tclparser 'parse' command API covering at least the subcommands and token shapes punk::lib consumes - with behavioural parity verified against the C library rather than assumed, and the C parser still preferred where present (performance). |
||||||
|
Acceptance: the punk::lib tclparser_tcl error stub is replaced by a working pure-Tcl implementation of the 'parse' subcommands punk::lib currently uses ('parse command' at minimum; the full covered set enumerated in this file during the work), returning the same parse-tree shapes (ranges, token types, expansion handling) the C library returns for those calls; a parity testsuite compares pure-Tcl output against the C tclparser across a recorded probe corpus (representative punkshell module scripts plus edge cases: {*} expansion, comments, backslash-newline continuation, nested command substitution, braces/quotes in words) - parity tests capability-gated on the binary, pure-Tcl-only tests running everywhere; a documented punk::lib analysis entry point (e.g. tclword_to_scriptlist) demonstrably works under a plain tclsh with no parser binary on the package path; dispatch prefers the C parser when available with the fallback engaging automatically otherwise; the reference source's location under TEMP_REFERENCE and its provenance (origin URL, version/checkin) are recorded in this file. |
||||||
|
|
||||||
|
## Context |
||||||
|
|
||||||
|
punk::lib uses the tclparser C library's 'parse command' for script analysis (word |
||||||
|
-> scriptlist conversion, argument/expansion classification - the G-019 |
||||||
|
dependency-scan direction). The binary ships only in the punk9win kit's lib_tcl9, |
||||||
|
so the analysis paths are unavailable on plain tclsh, other kits and other |
||||||
|
platforms - and lib-999999.0a1.0.tm carries an explicit placeholder: |
||||||
|
tclparser_tcl (currently an error stub telling the user to install the C library) |
||||||
|
plus a recorded deliberation at tclscript_to_toplevelinfo noting "we don't have a |
||||||
|
pure-tcl implementation of 'parse' available as a fallback". A tested pure-Tcl |
||||||
|
implementation removes the binary constraint from every current and future consumer |
||||||
|
and aligns with the G-004 no-committed-binaries direction. It also widens where the |
||||||
|
G-069 splitter cross-check lint can run. |
||||||
|
|
||||||
|
The user will place the latest obtainable tclparser source (C implementation and |
||||||
|
any docs/tests) under TEMP_REFERENCE as the read-only behavioural reference |
||||||
|
(candidate upstreams: the aspect fossil fork at chiselapp and the ActiveState |
||||||
|
teapot tree, both already noted in the tclparser_tcl stub comments; punkshell also |
||||||
|
carries a moduledoc for the parse command at |
||||||
|
src/modules/punk/args/moduledoc/parser-999999.0a1.0.tm). |
||||||
|
|
||||||
|
## Approach |
||||||
|
|
||||||
|
- Scope the API by consumption, not by the full tclparser surface: enumerate the |
||||||
|
subcommands and result fields punk::lib actually reads (parse command ranges, |
||||||
|
token lists, restRange handling, simple-expansion trees), implement those first, |
||||||
|
and record the covered/uncovered set here. Other subcommands (expr, varname, |
||||||
|
list) follow only if a consumer needs them. |
||||||
|
- Parity-first workflow (G-055 spirit): build the probe corpus and the |
||||||
|
compare-against-C harness before/alongside the implementation, so every |
||||||
|
implemented construct lands with a parity pin. Corpus draws on real punkshell |
||||||
|
module sources (rich in expansions, multi-line commands and comments) plus |
||||||
|
targeted edge cases. |
||||||
|
- Dispatch: a thin front (in punk::lib or the new module) that package-requires the |
||||||
|
C parser and falls back to the pure-Tcl engine; consumers call one entry point. |
||||||
|
Performance note recorded rather than chased - the C library stays preferred |
||||||
|
where installed. |
||||||
|
- Placement: the stub lives in punk::lib today; a parser is a sizeable, |
||||||
|
separately-testable unit, so a dedicated module (name TBD, e.g. under punk::) |
||||||
|
with punk::lib delegating is the likely shape - decided in the work. |
||||||
|
|
||||||
|
## Notes |
||||||
|
|
||||||
|
- Related: G-019 (dependency-scan module trimming - the main analysis consumer), |
||||||
|
G-004 (binary-free direction), G-069 (lint gains a binary-free substrate), |
||||||
|
G-055 (parity-verification workflow precedent). |
||||||
|
- The tclparser_tcl stub's partial char-loop sketch (in_dq/in_cb/escape state) is a |
||||||
|
starting point only; the reference source's Tcl_ParseCommand semantics are the |
||||||
|
contract. |
||||||
Loading…
Reference in new issue