From 5e3e3efa99631840fe4538e27b22d38be841cc79 Mon Sep 17 00:00:00 2001 From: Julian Noble Date: Mon, 6 Jul 2026 16:59:47 +1000 Subject: [PATCH] add goals G-018..G-021: build spectrum + screen capture/input automation G-018: zig-built plain zip-based tclsh kits with no punk infrastructure (Tcl 9 kit satisfies acceptance; 8.6 interim = plain non-kit builds, kits when practical, non-gating). Notes record that wish executables are never a build target - Tk ships as a loadable package. G-019: dependency-scan-driven module trimming for punk executables via the tclparser parse API (c-only library today; punk::lib pure-Tcl fallback is a stub), with a declared-extras manifest for scan-invisible dynamic requires; batteries-included builds preserved. G-020: screen capture + input injection module with per-platform backends - Windows first (treectrl loupe + twapi, all already shipped in punk9win kit); X11 (Linux/FreeBSD) and macOS follow behind the same contract; Wayland-native out of scope. Platform show-stopper assessment recorded in the detail file. G-021: agent-drivable visual/UI verification - window match, snapshot to PNG/base64, scripted interactions - through G-015 piped script calls, with guidance to land alongside G-017. aloupe.tcl stays a standalone app. Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.com --- GOALS.md | 24 ++++++ goals/G-018-zig-plain-tclsh-kits.md | 56 ++++++++++++ .../G-019-dependency-scan-module-trimming.md | 67 +++++++++++++++ goals/G-020-screencap-input-module.md | 86 +++++++++++++++++++ goals/G-021-agent-visual-verification.md | 55 ++++++++++++ 5 files changed, 288 insertions(+) create mode 100644 goals/G-018-zig-plain-tclsh-kits.md create mode 100644 goals/G-019-dependency-scan-module-trimming.md create mode 100644 goals/G-020-screencap-input-module.md create mode 100644 goals/G-021-agent-visual-verification.md diff --git a/GOALS.md b/GOALS.md index af445e3c..bb233838 100644 --- a/GOALS.md +++ b/GOALS.md @@ -150,3 +150,27 @@ Acceptance: with a git-only project on disk registered in the chosen enumeration Scope: AGENTS.md (root) or a child doc it indexes (guidance content only - no code) Goal: once G-015 makes piped script calls reliable, repository guidance directs agents asked to locate another local project to query it via a piped `projects.work` call to a punk executable instead of grepping/globbing the wider filesystem. Acceptance: root AGENTS.md (or a child doc indexed from it) records the exact recommended invocation - executable, subcommand, glob usage, expected output shape - and states when filesystem scanning remains appropriate (projects not registered in any discovery source); the guidance is added only after G-015 is achieved (and notes the fossil-only limitation until G-016); following the documented pattern, an agent locates a named sibling project's checkout dir with a single piped call. + +### G-018 [proposed] Zig-built plain tclsh kits: self-contained zip-based executables without punk infrastructure +Scope: build.zig / build.zig.zon (per G-005), src/runtime/, src/make.tcl integration +Detail: goals/G-018-zig-plain-tclsh-kits.md +Goal: developers can use the G-005 zig build system to produce self-contained zip-based tclsh executables that carry no punk-specific infrastructure (no punk boot layer, punk modules, or punk apps) - plain tclsh kits usable independently of the punkshell product. +Acceptance: a documented zig invocation on a clean checkout (zig toolchain available) produces a zip-based tclsh executable for at least one target platform that runs conventional tclsh invocations (` script.tcl args`, piped stdin) on a machine with no Tcl installation; a listing of the kit's mounted/zip contents shows stock Tcl (plus any declared stock runtime deps) and no punk namespaces, punk boot files, or punkshell apps; the punk-flavoured executables remain producible alongside. + +### G-019 [proposed] Dependency-scan-driven module trimming for punk-based executables +Scope: src/make.tcl, src/modules/punk/lib-999999.0a1.0.tm (tclparser use), src/vfs/ (kit assembly), scanning module (new or existing punk module - to be determined) +Detail: goals/G-019-dependency-scan-module-trimming.md +Goal: a package-dependency scan from an executable's entrypoint (candidate basis: the tclparser parse API - currently satisfied only by the c-only tclparser library, with punk::lib's pure-Tcl fallback an unimplemented stub) determines the module closure the executable actually requires, so a build can ship only those modules - while 'batteries included' builds remain a supported alternative, not a casualty. +Acceptance: for at least one punk-based executable target, the build can run a dependency scan from its entrypoint producing the closure of required packages/modules plus a mechanism to declare dynamically-loaded extras the scan cannot see; a trimmed kit assembled from that closure starts and passes its basic function check (e.g. repl launch or the app's smoke test) with no missing-package errors; the trimmed kit's module listing is a strict subset of the batteries-included equivalent (demonstrating real exclusion); batteries-included builds remain producible unchanged. + +### G-020 [proposed] Screen capture and input injection module with per-platform backends (Windows first) +Scope: src/modules/punk/ (new module - name TBD), src/vfs/punk9win.vfs/lib_tcl9/ (existing treectrl/Img/twapi payloads); scriptlib/aloupe.tcl stays untouched as a standalone app +Detail: goals/G-020-screencap-input-module.md +Goal: a punk module drives screen/window capture and mouse/keyboard injection from scripts via per-platform backends - Windows (treectrl loupe capture + twapi input/window-location) is the initial complete target, with the backend contract designed so X11 (Linux/FreeBSD) and macOS backends can be added without changing callers, and Wayland-native sessions explicitly out of scope. +Acceptance: on Windows from a punk shell or script: a screen region and a window located by title/class pattern are each captured to a Tk photo and written as a valid PNG; mouse movement/click and key events injected into a located test window produce their observable effect (typed text arrives, click acts); window location returns the handle and geometry for a pattern; a capability-introspection call reports per-feature support and an unsupported platform/backend yields a clean capability-based refusal, not a crash; the backend interface (capture / input / window-locate) is documented well enough that a non-Windows backend can be added without modifying callers; the aloupe script remains functional and unmodified. + +### G-021 [proposed] Agent-drivable visual/UI verification via piped snapshot and interaction calls +Scope: src/modules/punk/ (G-020 module's agent-facing surface), AGENTS.md guidance (post G-015 pattern), src/tests/ (visual-verification test hooks) +Detail: goals/G-021-agent-visual-verification.md +Goal: a tool-calling agent can, during a session, use piped script calls (G-015) to a punk executable to locate the applicable UI window, snapshot it to a PNG file and/or base64 output suitable for AI image analysis, and drive mouse/keyboard interactions - enabling tests whose verification is visual-only and/or input-driven. +Acceptance: on Windows, single piped script calls (no interactive session) can: list/match windows for a pattern with machine-parseable output; save a located window's snapshot to a caller-specified path and optionally emit it base64 on stdout; run a scripted interaction sequence (focus, click at offset, type text, snapshot) end-to-end; failures exit nonzero with the error on stderr per G-015 semantics; the invocation patterns are documented for agents alongside the G-017 guidance; at least one real visual-or-input-driven verification (e.g. a Tk app smoke test) is exercised through this path. diff --git a/goals/G-018-zig-plain-tclsh-kits.md b/goals/G-018-zig-plain-tclsh-kits.md new file mode 100644 index 00000000..066a7d0e --- /dev/null +++ b/goals/G-018-zig-plain-tclsh-kits.md @@ -0,0 +1,56 @@ +# G-018 Zig-built plain tclsh kits: self-contained zip-based executables without punk infrastructure + +Status: proposed +Scope: build.zig / build.zig.zon (per G-005), src/runtime/, src/make.tcl integration +Acceptance: as in root GOALS.md index (canonical). + +## Context + +G-005 establishes zig-based build infrastructure to build binary dependencies +(including Tcl9) from source. This goal extends that infrastructure's output +targets: beyond building the punkshell product's own runtimes, developers should +be able to produce plain, self-contained, zip-based tclsh executables with no +punk-specific infrastructure at all - no punk boot layer (punk_main.tcl +dispatch), no punk modules, no punkshell apps. A general-purpose single-file +tclsh, useful independently of punkshell. + +This defines one end of a deliberate spectrum of buildable executables: + +- plain tclsh kit (this goal) - stock Tcl only +- trimmed punk executable (G-019) - punk infrastructure plus only the modules a + dependency scan proves needed +- 'batteries included' punk executable - the current full-kit approach + +## Approach + +- Zip-based (zipfs) kit assembly as the packaging mechanism, consistent with + Tcl 9's built-in zipfs support. +- **Tcl 8.6 phasing (decided 2026-07-06)**: the 8.6 kit mechanism is more + complex to support (no built-in zipfs) and is wanted in the end, but must not + block other work. Interim step: support plain (non-kit) tcl 8.6 builds from + the zig infrastructure, and implement 8.6 kits when practical. This goal's + acceptance is satisfiable by a Tcl 9 kit alone; 8.6 kit support is a + follow-on within the same goal's spirit, not a gate. +- The plain kit is a natural first smoke target for G-005's Tcl-from-source + build: it exercises the compiled runtime with minimal packaging logic before + the more elaborate punk kit assembly. +- `src/make.tcl` integration so the target is invocable through the normal + project build entrypoint as well as directly via zig. + +## Alternatives considered + +- Relying on third-party tclkit/kitcreator builds for plain kits - rejected as + the durable answer: the point of G-005 is developer-reproducible builds from + source in this project's toolchain, and external kit builders are exactly the + committed-binary/procurement dependency G-004..G-006 are removing. + +## Notes + +- Depends on G-005 (zig builds Tcl from source); complements G-004 (no committed + binaries) by making plain runtimes reproducible on demand. +- 'Self-contained' means: runs conventional tclsh invocations on a machine with + no Tcl installation - stock Tcl library scripts ride inside the kit. +- Wish executables are not a build target at any point on the spectrum: kits + that want Tk ship it as a loadable package (as the punk Windows kit already + does - `package require Tk` from the tclsh-style executable, console + retained). Applies to plain kits here and punk kits alike. diff --git a/goals/G-019-dependency-scan-module-trimming.md b/goals/G-019-dependency-scan-module-trimming.md new file mode 100644 index 00000000..98e1184b --- /dev/null +++ b/goals/G-019-dependency-scan-module-trimming.md @@ -0,0 +1,67 @@ +# G-019 Dependency-scan-driven module trimming for punk-based executables + +Status: proposed +Scope: src/make.tcl, src/modules/punk/lib-999999.0a1.0.tm (tclparser use), src/vfs/ (kit assembly), scanning module (new or existing punk module - to be determined) +Acceptance: as in root GOALS.md index (canonical). + +## Context + +Punk executables are currently 'batteries included' - the kit assembly ships the +full module snapshot regardless of what a given executable's app actually uses. +For executables built around a specific app or entrypoint, a package-dependency +scan could determine the module closure genuinely required and let the build +ship only that, giving smaller artifacts. Batteries-included builds remain a +supported alternative (the middle and far points of the spectrum described in +G-018's context section). + +The candidate scanning basis is static parsing of Tcl source via the tclparser +`parse` API (tclpro-descended; https://wiki.tcl-lang.org/page/tclparser). Its +current availability shapes the work: + +- The real implementation is the **c-only tclparser library** + (`package require parser`); punk::lib uses it opportunistically. +- punk::lib carries a same-API pure-Tcl fallback, `punk::lib::tclparser_tcl` + (src/modules/punk/lib-999999.0a1.0.tm ~3322), which is an unimplemented stub + that errors advising installation of the C library. +- punk::args::moduledoc::parser documents the parse API. + +Using the C library inside build tooling makes the build depend on a native +binary - intersecting the G-004/G-005/G-006 binary-provenance story (zig-built +or consent-downloaded, never committed). Completing the pure-Tcl fallback would +remove that coupling at some accuracy/performance cost. The choice (or ordering +- e.g. C library first, fallback later) is an implementation decision to record +here. + +## Approach + +- Static scan from the executable's entrypoint/app: walk `package require` + (and `tcl::tm` / source-time load constructs) through the reachable sources, + resolving the transitive closure against the project's module trees. +- **Dynamic requires are the known limit**: computed package names, plugin-style + loading, and `package require` inside rarely-hit code paths cannot be proven + statically. A declaration mechanism for extras (per-executable manifest of + packages to include beyond the scan result) is part of the goal's acceptance, + not an afterthought. +- Scan output should be a reviewable artifact (the closure list), feeding kit + assembly rather than silently filtering it - so a wrong exclusion is + diagnosable from the build record. +- Verification of a trimmed kit is behavioural (starts, passes its app's smoke + check) plus structural (module listing strictly a subset of the + batteries-included equivalent). + +## Alternatives considered + +- Runtime tracing (run the app, record `package require` calls) instead of + static parsing - rejected as primary: coverage-dependent (only exercised + paths are seen), though it could later complement the static scan for + validating the declared-extras list. +- Naive regex scan for `package require` lines - rejected: misses computed + names it can't even flag, false-positives in strings/comments; the tclparser + parse API exists precisely to do this properly. + +## Notes + +- Interacts with G-005/G-006 (how the c-only tclparser binary is provisioned if + chosen) and G-018 (the spectrum of executable flavours). +- punk::lib sites already noting tclparser use/fallback: tclword_to_scriptlist + and related parsing helpers (~lib 3322-3580). diff --git a/goals/G-020-screencap-input-module.md b/goals/G-020-screencap-input-module.md new file mode 100644 index 00000000..a0c728d0 --- /dev/null +++ b/goals/G-020-screencap-input-module.md @@ -0,0 +1,86 @@ +# G-020 Screen capture and input injection module with per-platform backends (Windows first) + +Status: proposed +Scope: src/modules/punk/ (new module - name TBD), src/vfs/punk9win.vfs/lib_tcl9/ (existing treectrl/Img/twapi payloads); scriptlib/aloupe.tcl stays untouched as a standalone app +Acceptance: as in root GOALS.md index (canonical). + +## Context + +Punk already runs Tk from tclsh-style kits with the console retained (Tk is a +loadable package in the kit; the repl event loop services it - no wish +executable involved, and wish binaries are not a build target). The standalone +`scriptlib/aloupe.tcl` (aloupe 1.8.2, MIT) demonstrates the capture primitive: +treectrl's `loupe` command copies an arbitrary screen region into a Tk photo +(aloupe.tcl ~454), with Img for extra file formats. The Windows kit already +ships every needed binary: `src/vfs/punk9win.vfs/lib_tcl9/` has tk9.0, +treectrl 2.5.2 (tcl9treectrl252.dll), Img 2.0.1/2.1.0, twapi 5.0.2, and +cffi 2.0.3. + +This goal turns that demonstration into a scriptable module: capture plus +mouse/keyboard injection plus window location, so scripts (and, via G-021, +agents) can drive and observe GUI state. aloupe.tcl remains a standalone app; +the module is a separate development, not a refactor of it. + +## Platform assessment (investigated 2026-07-06) + +- **Windows (initial primary target)** - no show-stoppers, all deps already in + the kit. Capture: treectrl loupe (GDI). Input: twapi `send_input` + (SendInput). Window locate: twapi enumeration/geometry/foreground. Gotchas to + handle/document: desktop session must be unlocked and interactive (locked + workstation / disconnected RDP -> black captures, failed injection); + per-monitor DPI scaling affects coordinate mapping; UIPI blocks injecting + into elevated windows from a non-elevated process; multi-monitor negative + coordinates. +- **Linux (X11)** - supported: treectrl loupe X11 impl (XGetImage); injection + via XTest (xdotool external tool, or cffi binding to libXtst); locate via + xdotool/wmctrl. Xvfb gives a fully headless mode - the preferred arrangement + for automated visual verification. +- **Wayland-native - OUT OF SCOPE (decided 2026-07-06)**: XGetImage cannot see + native Wayland surfaces and XTest cannot inject into them by design; the + escape hatches (xdg-desktop-portal Screenshot/ScreenCast D-Bus APIs, ydotool + via uinput) are permission-heavy with no ready Tcl bindings. Revisit via + portals if/when needed; X11/Xvfb is the supported Linux mode. +- **FreeBSD** - same X11 story; tktreectrl/tkimg/xdotool in ports; twapi N/A. +- **macOS** - supportable, permission-gated: TCC requires Screen Recording + (capture) and Accessibility (injection) grants to the hosting process - + interactive one-time grants, awkward for unattended CI. treectrl loupe aqua + support is unverified and non-blocking: the built-in `screencapture` CLI + (region/window-id to file) is the natural capture backend; injection options + are osascript/System Events, cliclick (third-party), or cffi -> CGEventPost. + `src/vfs/macosx-arm/` already exists in the tree. + +## Approach + +- Three backend roles behind one API: **capture**, **input**, **window-locate**. + Per-platform backend selection with capability introspection - callers ask + what is supported rather than probing by failure. Echoes the G-001 pluggable + pattern. +- Windows backend first and complete (acceptance gates on it). X11 and macOS + backends are in-scope follow-ons behind the same contract; their absence on a + platform must produce a clean capability-based refusal. +- Output normalization: capture lands in a Tk photo; PNG via Tk's core photo + support (Img optional for other formats). File save and in-memory (base64) + both supported - the G-021 consumer needs both. +- cffi (bundled) is the escape hatch for gaps without new C extensions - e.g. + Windows PrintWindow for capturing occluded windows (region-based loupe cannot), + libXtst on X11 if avoiding the xdotool external-tool dependency, CGEventPost + on macOS. +- Binary deps (treectrl, Img, twapi, cffi) follow the G-004/G-005/G-006 + provisioning story on each platform; nothing new is committed to the repo. + +## Alternatives considered + +- Refactoring aloupe.tcl into the module - rejected: it is a vendored-style + standalone app under user-only scriptlib/; keeping it untouched preserves a + known-working reference implementation. +- Pure-twapi capture on Windows (skip treectrl) - not pursued initially: twapi + has no ready BitBlt/screen-grab wrapper; treectrl loupe is already shipped + and proven. cffi/PrintWindow may supplement later for occluded windows. +- Wayland-native support via portals/ydotool - out of scope (see platform + assessment); recorded for possible future revisit. + +## Notes + +- Related: G-018 (kit composition; Tk as loadable package, no wish binaries), + G-019 (a trimmed capture-capable executable is a plausible scan target), + G-021 (agent-facing surface over this module), G-001 (backend plugin pattern). diff --git a/goals/G-021-agent-visual-verification.md b/goals/G-021-agent-visual-verification.md new file mode 100644 index 00000000..6d91bc1e --- /dev/null +++ b/goals/G-021-agent-visual-verification.md @@ -0,0 +1,55 @@ +# G-021 Agent-drivable visual/UI verification via piped snapshot and interaction calls + +Status: proposed +Scope: src/modules/punk/ (G-020 module's agent-facing surface), AGENTS.md guidance (post G-015 pattern), src/tests/ (visual-verification test hooks) +Acceptance: as in root GOALS.md index (canonical). + +## Context + +Some behaviours can only be verified visually (rendering, theming, layout, +ANSI-vs-Tk display parity) or by driving real input (mouse/keyboard-dependent +flows). Today an agent in a session has no sanctioned way to see or drive GUI +state. With G-020 providing capture/input/window-location as a module and +G-015 providing reliable piped one-shot script calls, the combination lets a +tool-calling agent: determine the applicable UI window, snapshot it, analyze +the image (agents are multimodal), and drive interactions - all from ordinary +tool calls during a session. + +## Approach + +- Thin agent-facing surface over the G-020 module, exercised through the G-015 + `script` subcommand: each operation is a single piped call with + machine-parseable output and G-015 exit-code semantics (nonzero + stderr on + failure, never an interactive shell). +- Operations: window list/match by pattern; snapshot a located window or region + to a caller-specified PNG path; optional base64-on-stdout emission for direct + image ingestion by the calling agent; scripted interaction sequences (focus, + click at offset, key/type, snapshot) composable in one piped script. +- Windows first, per G-020. Capability introspection flows through, so an agent + on an unsupported platform gets a clean "not supported here" rather than a + hang or crash. +- Documentation for agents lands alongside the G-017 guidance (the same "use + the punk mechanism, not ad-hoc scanning/screenshotting" pattern): exact + invocations, output shapes, and the session prerequisites (unlocked desktop, + DPI notes) from G-020's platform assessment. +- At least one real verification consumer in src/tests/ or a runbook (e.g. a + Tk app smoke test asserting on a snapshot) proves the loop end-to-end. + +## Alternatives considered + +- OS-level screenshot tooling driven directly by agents (PowerShell/GDI + one-liners, screencapture, import/scrot) - rejected as the sanctioned path: + per-platform incantations with no window-location or input story, no + capability model, and nothing reusable by punk scripts; the punk module route + gives one contract everywhere punk runs. +- Building agent tooling (MCP server or similar) directly into punk now - + deferred: piped G-015 calls already fit existing agent Bash/exec tools; a + richer integration can layer later without changing the module. + +## Notes + +- Depends on G-020 (mechanism) and G-015 (reliable piped calls); guidance + placement follows G-017. +- Image handoff sizing: agents ingest images with dimension/size limits; + snapshot operations should support downscale/crop options eventually - + acceptance keeps to save + base64, with sizing noted as a natural follow-on.