Browse Source
Both user-approved drafts arising from the G-045 tclparser assessment: - G-069: dev-time lint cross-checking punk::args record-splitter boundaries against tclparser parse command where the binary is available - dialect-aware comparison (-& pre-join, ANSI strip), capability-gated, zero new runtime dependencies for punk::args. - G-070: pure-Tcl implementation of the tclparser parse-command API scoped to what punk::lib consumes, parity-verified against the C library over a probe corpus; replaces the tclparser_tcl error stub with prefer-C/fallback-pure dispatch. Reference source to be user-provided under TEMP_REFERENCE with recorded provenance. Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.commaster
3 changed files with 114 additions and 0 deletions
@ -0,0 +1,50 @@
|
||||
# G-069 Dev-time lint: cross-check punk::args record splitting against tclparser where the binary is available |
||||
|
||||
Status: proposed |
||||
Scope: lint surface (location settled in the work: punk::args dev helper, dev commandset, or scriptlib/developer script), src/modules/punk/args-999999.0a1.0.tm (split_definition_records as consumed, no new dependency), src/tests/modules/punk/args/ (capability-gated suite) |
||||
Goal: a dev-time lint cross-checks the record boundaries produced by punk::args' definition record splitter (private::split_definition_records) against the tclparser C library's 'parse command' tokenization of the same (ANSI-stripped, dialect-adjusted) text, so drift between the definition dialect and real Tcl parsing rules is caught as the dialect grows (-&, @normalize, future continuation/normalization directives) - without punk::args itself gaining any dependency on the tclparser binary. |
||||
Acceptance: a documented lint command (location decided in the work) accepts definition text and/or registered definition ids and reports record-boundary divergence between split_definition_records and tclparser 'parse command'; the comparison accounts for documented dialect features rather than flagging them (at minimum: ANSI escapes stripped before parse, -& record continuation pre-joined or classified as expected divergence); the lint is capability-gated - a clean, explicit skip/notice when the tclparser package is unavailable, never an error - and `package require punk::args` continues to succeed with no tclparser present; a run across all definitions registered in a punk9win kit shell (where the tclparser binary ships) yields zero unexplained divergences, with any real divergences found either fixed or recorded in this file; exercised by a capability-gated test or a documented manual run recorded here. |
||||
|
||||
## Context |
||||
|
||||
While implementing the -& record-continuation token (G-045, archived), tclparser's |
||||
'parse' command was considered and rejected as the splitter's parsing substrate: the |
||||
definition dialect legitimately deviates from Tcl (unbalanced-bracket ANSI escapes as |
||||
data inside quoted values, line-based base-indent semantics, the -& token), and a |
||||
binary dependency in punk::args - the module that must load everywhere, including |
||||
bootsupport contexts and plain tclsh - inverts the G-004 direction. The full analysis |
||||
is recorded in goals/archive/G-045-punkargs-authoring-ergonomics.md (Progress, |
||||
increment 3) and in the punk::args 0.7.0 changelog. |
||||
|
||||
The rejected option still has value as a diagnostic: records are shaped like Tcl |
||||
commands (newline-delimited, quote/brace/backslash continuation), so where the binary |
||||
is available, parse-command boundaries are an independent oracle for the splitter. |
||||
As the dialect grows, an automated cross-check catches unintended divergence early - |
||||
the splitter's own testsuites (recordcontinuation.test, normalize.test, defquoting.test, |
||||
rendering.test) pin known behaviour but cannot flag novel drift against Tcl semantics. |
||||
|
||||
## Approach |
||||
|
||||
- Comparison pipeline: take the definition text as the splitter receives it, produce |
||||
splitter records; separately ansistrip the text, pre-join -& continuations (or |
||||
classify them), then iterate tclparser 'parse command' to produce boundary ranges; |
||||
compare record counts and boundary line positions, reporting divergences with |
||||
context. |
||||
- Corpus: registered definitions via punk::args introspection (update_definitions + |
||||
raw defs), plus ad-hoc text input for testing definition snippets during dialect |
||||
work. |
||||
- Home: this is developer tooling, not runtime behaviour - candidates are a |
||||
punk::args dev/diagnostic namespace proc (loaded lazily), a dev commandset command, |
||||
or a scriptlib/developer script following goals_lint.tcl precedent. Decide in the |
||||
work; the constraint is only that punk::args' runtime footprint is unchanged. |
||||
- The tclparser binary currently ships only in the punk9win kit (lib_tcl9); the lint |
||||
is expected to run there (or any environment with the package installed). If G-070 |
||||
(pure-Tcl tclparser) lands, the gate widens automatically if the lint requires |
||||
'parser' via ordinary package resolution. |
||||
|
||||
## Notes |
||||
|
||||
- Related: G-045 (archived - origin of the rejected-then-repurposed idea and the |
||||
dialect features the comparison must understand), G-070 (pure-Tcl tclparser would |
||||
let the lint run without the binary), G-004 (why the runtime must stay |
||||
dependency-free). |
||||
@ -0,0 +1,56 @@
|
||||
# G-070 Pure-Tcl tclparser: parse-command API fallback with behavioural parity against the C library |
||||
|
||||
Status: proposed |
||||
Scope: src/modules/punk/lib-999999.0a1.0.tm (tclparser_tcl stub + dispatch; new module if size warrants - decided in the work), src/tests/modules/punk/lib/ (parity + fallback suites), TEMP_REFERENCE/ (tclparser reference source, user-provided, read-only) |
||||
Goal: punk::lib's script-analysis machinery (tclword_to_scriptlist and the dependent analysis paths) runs on runtimes without the tclparser C binary, via a pure-Tcl implementation of the tclparser 'parse' command API covering at least the subcommands and token shapes punk::lib consumes - with behavioural parity verified against the C library rather than assumed, and the C parser still preferred where present (performance). |
||||
Acceptance: the punk::lib tclparser_tcl error stub is replaced by a working pure-Tcl implementation of the 'parse' subcommands punk::lib currently uses ('parse command' at minimum; the full covered set enumerated in this file during the work), returning the same parse-tree shapes (ranges, token types, expansion handling) the C library returns for those calls; a parity testsuite compares pure-Tcl output against the C tclparser across a recorded probe corpus (representative punkshell module scripts plus edge cases: {*} expansion, comments, backslash-newline continuation, nested command substitution, braces/quotes in words) - parity tests capability-gated on the binary, pure-Tcl-only tests running everywhere; a documented punk::lib analysis entry point (e.g. tclword_to_scriptlist) demonstrably works under a plain tclsh with no parser binary on the package path; dispatch prefers the C parser when available with the fallback engaging automatically otherwise; the reference source's location under TEMP_REFERENCE and its provenance (origin URL, version/checkin) are recorded in this file. |
||||
|
||||
## Context |
||||
|
||||
punk::lib uses the tclparser C library's 'parse command' for script analysis (word |
||||
-> scriptlist conversion, argument/expansion classification - the G-019 |
||||
dependency-scan direction). The binary ships only in the punk9win kit's lib_tcl9, |
||||
so the analysis paths are unavailable on plain tclsh, other kits and other |
||||
platforms - and lib-999999.0a1.0.tm carries an explicit placeholder: |
||||
tclparser_tcl (currently an error stub telling the user to install the C library) |
||||
plus a recorded deliberation at tclscript_to_toplevelinfo noting "we don't have a |
||||
pure-tcl implementation of 'parse' available as a fallback". A tested pure-Tcl |
||||
implementation removes the binary constraint from every current and future consumer |
||||
and aligns with the G-004 no-committed-binaries direction. It also widens where the |
||||
G-069 splitter cross-check lint can run. |
||||
|
||||
The user will place the latest obtainable tclparser source (C implementation and |
||||
any docs/tests) under TEMP_REFERENCE as the read-only behavioural reference |
||||
(candidate upstreams: the aspect fossil fork at chiselapp and the ActiveState |
||||
teapot tree, both already noted in the tclparser_tcl stub comments; punkshell also |
||||
carries a moduledoc for the parse command at |
||||
src/modules/punk/args/moduledoc/parser-999999.0a1.0.tm). |
||||
|
||||
## Approach |
||||
|
||||
- Scope the API by consumption, not by the full tclparser surface: enumerate the |
||||
subcommands and result fields punk::lib actually reads (parse command ranges, |
||||
token lists, restRange handling, simple-expansion trees), implement those first, |
||||
and record the covered/uncovered set here. Other subcommands (expr, varname, |
||||
list) follow only if a consumer needs them. |
||||
- Parity-first workflow (G-055 spirit): build the probe corpus and the |
||||
compare-against-C harness before/alongside the implementation, so every |
||||
implemented construct lands with a parity pin. Corpus draws on real punkshell |
||||
module sources (rich in expansions, multi-line commands and comments) plus |
||||
targeted edge cases. |
||||
- Dispatch: a thin front (in punk::lib or the new module) that package-requires the |
||||
C parser and falls back to the pure-Tcl engine; consumers call one entry point. |
||||
Performance note recorded rather than chased - the C library stays preferred |
||||
where installed. |
||||
- Placement: the stub lives in punk::lib today; a parser is a sizeable, |
||||
separately-testable unit, so a dedicated module (name TBD, e.g. under punk::) |
||||
with punk::lib delegating is the likely shape - decided in the work. |
||||
|
||||
## Notes |
||||
|
||||
- Related: G-019 (dependency-scan module trimming - the main analysis consumer), |
||||
G-004 (binary-free direction), G-069 (lint gains a binary-free substrate), |
||||
G-055 (parity-verification workflow precedent). |
||||
- The tclparser_tcl stub's partial char-loop sketch (in_dq/in_cb/escape state) is a |
||||
starting point only; the reference source's Tcl_ParseCommand semantics are the |
||||
contract. |
||||
Loading…
Reference in new issue