Compare commits
6 Commits
fbc7654c5b
...
983cb44f55
| Author | SHA1 | Date |
|---|---|---|
 Julian Noble
|
983cb44f55 | 1 month ago |
|
Julian Noble
|
561169280f | 1 month ago |
|
Julian Noble
|
64d0962ad6 | 1 month ago |
|
Julian Noble
|
1ddaa0d03f | 1 month ago |
|
Julian Noble
|
3f209edec4 | 1 month ago |
|
Julian Noble
|
036d4c0dec | 1 month ago |
13 changed files with 499 additions and 27 deletions
@ -0,0 +1,21 @@
|
||||
# Achieved Goals Archive |
||||
|
||||
This file holds one-line records of goals that have been achieved and moved out of the active `GOALS.md` index to keep that file lean. Records here are historical context only — they explain why code exists in its current shape and are useful when future agents refactor or revisit the same area. |
||||
|
||||
## Format |
||||
|
||||
Each archived goal is one line, preserving its original ID and acceptance criterion so it remains traceable: |
||||
|
||||
``` |
||||
### G-<id> [achieved <YYYY-MM-DD>] <short title> → detail: goals/archive/G-<id>-<slug>.md |
||||
Scope: <as in original index> |
||||
Acceptance: <as in original index> |
||||
``` |
||||
|
||||
If a goal had no detail file, omit the `→ detail:` clause. |
||||
|
||||
Do not edit archived entries except to fix a broken path. If an archived goal is reopened, move it back to `GOALS.md` with a new ID and mark the old entry `superseded by G-<new id>`. |
||||
|
||||
## Archived goals |
||||
|
||||
_None yet._ |
||||
@ -0,0 +1,87 @@
|
||||
# Project Goals |
||||
|
||||
This file is the canonical, harness-agnostic index of technical project goals for ShellSpy. It is referenced from the root `AGENTS.md` Child DOX Index and is a required read for any non-trivial work, so that agents can discover goals whose scope intersects their work. |
||||
|
||||
Detail prose for goals that need it lives in `goals/G-<id>-<slug>.md` (see `goals/AGENTS.md`). The index entry is canonical; a detail file only elaborates and never contradicts its index entry. |
||||
|
||||
## Format |
||||
|
||||
Each goal is one block: |
||||
|
||||
``` |
||||
### G-<id> [<status>] <short title> |
||||
Scope: <repo paths or module areas this goal touches> |
||||
Goal: <one line — what done looks like, self-contained> |
||||
Detail: goals/G-<id>-<slug>.md <- optional, omit if absent |
||||
Acceptance: <measurable, verifiable pass/fail criterion> |
||||
``` |
||||
|
||||
### Status tags |
||||
- `proposed` — not yet started; awaiting user confirmation to go `active` |
||||
- `active` — in progress |
||||
- `achieved <YYYY-MM-DD>` — done; kept as a one-line record |
||||
- `abandoned` — dropped; one line on why stays in the entry |
||||
- `superseded by G-<id>` — replaced; do not delete the old entry |
||||
|
||||
### Maintenance rules |
||||
- Only the user adds or edits goals. Agents must not invent goals. |
||||
- An agent whose work satisfies a goal's `Acceptance:` must flip that goal to `achieved <date>` as part of its DOX closeout pass, and report the flip in its completion summary. |
||||
- Agents must not flip `proposed` → `active`. They flag it in their completion report for the user to confirm. |
||||
- The `Goal:` line must stay self-contained enough that an agent who skips the detail file still does no harm. Detail files are enrichment, not load-bearing for safety. |
||||
- When the achieved section grows past ~30 entries, the oldest are moved to `GOALS-archive.md` and their detail files to `goals/archive/`. |
||||
- If a goal cannot be safely summarized in one line, that is a signal it is really two goals — split it. |
||||
|
||||
## Authoring a new goal (for agents) |
||||
|
||||
When the user asks to "write a goal for X" or "help me draft a goal for Y", do the following: |
||||
|
||||
1. Read this file in full so you know the format and can pick the next free `G-<id>`. |
||||
2. Ask the user only the questions below. Do not invent answers; ask them one at a time or batched if the user prefers. Stop asking once every required field has a real answer. |
||||
- **Scope:** Which repo paths or module areas does this goal touch? (paths are preferred; module names are acceptable if paths are not yet known) |
||||
- **Goal:** In one sentence, what does done look like? Push for an outcome, not an activity ("X compiles to bytecode ≤ 1.10× cost of Y", not "improve compiler performance"). |
||||
- **Acceptance:** What is the measurable, verifiable pass/fail criterion an agent can check against? If the user cannot state one, propose 2-3 candidate criteria and ask them to pick or refine. |
||||
- **Status:** Default to `proposed` unless the user says it is already in progress (`active`). |
||||
- **Detail file?** Only if the goal has non-obvious rationale, a multi-phase plan, alternatives worth recording, or needs more than ~3 lines of prose to state properly. If yes, propose a slug and offer to draft the detail file too. If no, omit the `Detail:` line. |
||||
3. Draft the goal block in this file's format and show it to the user for review and edit. Do not commit it as `active` unless the user confirms. |
||||
4. If a detail file is warranted and the user approves, create `goals/G-<id>-<slug>.md` using the structure in `goals/AGENTS.md`. |
||||
5. Do not delete or rewrite existing goals to make room for a new one. Append with the next free ID. |
||||
|
||||
## Goals |
||||
|
||||
<!-- Append new goals below using the format above. Keep the list ordered by G-<id>. --> |
||||
|
||||
### G-001 [proposed] Pluggable console backends for non-detectable terminals |
||||
Scope: src/modules/opunk/console-999999.0a1.0.tm, src/modules/punk/console-999999.0a1.0.tm, src/modules/punk/repl-999999.0a1.0.tm, src/lib/app-punkshell/punkshell.tcl |
||||
Detail: goals/G-001-pluggable-console-backends.md |
||||
Goal: an interactive REPL can be launched against a non-detectable terminal-like device (ssh channel, tk text widget) via an ::opunk::Console subclass, with no edits to the base class or punk::console. |
||||
Acceptance: a subshell started with an ssh-channel-backed and a tk-widget-backed ::opunk::Console subclass runs an interactive REPL that reads/writes through that console; size, at_eof, and can_respond are answered by the subclass overrides; the base ::opunk::Console and punk::console module are unchanged. |
||||
|
||||
### G-002 [proposed] Non-nested subshell with console targeting and inter-subshell comms |
||||
Scope: src/modules/punk/repl-999999.0a1.0.tm, src/modules/punk/repl/codethread-999999.0a1.0.tm |
||||
Detail: goals/G-002-non-nested-subshell.md |
||||
Goal: a subshell can target a named console (default or non-default) and run without blocking the parent, replacing the synchronous nested interp-eval model. |
||||
Acceptance: a parent REPL launches a subshell against a named console and continues processing its own input while the subshell runs; the parent can signal/query the running subshell; thread::send -async dispatched from within the subshell's code interp arrives at that interp (so packages like promise work when thread features aren't disabled); the "first subshell asymmetry" TODO at repl-999999.0a1.0.tm:3130 is resolved; existing synchronous `subshell punk`/`safe`/`safebase`/`punksafe` behaviour is preserved as a default mode. |
||||
|
||||
### G-003 [proposed] Configurable resource limits and sandboxing on subshell interps |
||||
Scope: src/modules/punk/repl-999999.0a1.0.tm, src/modules/punk/repl/codethread-999999.0a1.0.tm |
||||
Detail: goals/G-003-subshell-resource-limits.md |
||||
Goal: a subshell's code interp can be launched with configurable resource limits (command-count, time) and sandboxing features, building on the resolved first-subshell asymmetry from G-002. |
||||
Acceptance: a subshell can be launched with at least one resource limit (command-count via `interp limit -command`, or time via `interp limit -time`) and one sandboxing feature (e.g. `interp hide` of a command, or full safe-interp restrictions) applied to its code interp, enforceable regardless of subshell nesting depth; a subshell can be configured anywhere on the spectrum from unrestricted to fully safe via expose/hide of commands; the existing default subshell behaviour (no limits, no extra sandbox beyond the existing safe/safebase/punksafe types) is preserved when no limits are configured. |
||||
|
||||
### G-004 [proposed] No executable binaries committed to the repository |
||||
Scope: repo-wide (bin/, src/vfs/, src/vendorlib/, src/vendormodules/, src/bootsupport/) |
||||
Detail: goals/G-004-no-committed-binaries.md |
||||
Goal: the committed repository contains no executable binaries; zip-based .tm modules are allowed but not if they embed executables. |
||||
Acceptance: a scan of the committed tree finds no executable binaries (shared libs, .exe, native .so/.dll/.dylib, bare ELF/Mach-O); any zip-based .tm modules present contain no embedded executables; the binary artifacts previously committed are retrievable via G-005 (build from source) or G-006 (pre-built download) so their removal does not break builds. |
||||
|
||||
### G-005 [proposed] Zig-based build infrastructure for binary dependencies from source |
||||
Scope: src/runtime/, build.zig / build.zig.zon (new), src/make.tcl integration |
||||
Detail: goals/G-005-zig-build-infrastructure.md |
||||
Goal: a zig-based build system retrieves and builds binary dependencies (including Tcl9) from source, replacing the committed-binary approach for the vendored/native components. |
||||
Acceptance: running the zig build produces the binary artifacts the repo previously committed (at minimum: Tcl9 library for one target platform); existing Tcl9-zig experiments brought into the project; `tclsh src/make.tcl` integrates with the zig build so a normal project build retrieves/builds binaries via zig when not present; no binary artifacts need to be committed for the build to succeed on a clean checkout with the zig toolchain available. |
||||
|
||||
### G-006 [proposed] Optional pre-built binary artifact download with consent gating |
||||
Scope: src/runtime/, src/make.tcl integration, user-config (consent flags) |
||||
Detail: goals/G-006-prebuilt-artifact-download.md |
||||
Goal: pre-built binary artifacts can be downloaded from a separate related binary-artifacts repository or user-configured sources, gated by explicit user consent/configuration by default. |
||||
Acceptance: a download mechanism fetches binary artifacts (the same set the zig build produces) from a configured source on demand; by default the download is gated behind explicit user consent (a config flag or interactive prompt) and does not occur silently; a user-configured source URL overrides the default binary-artifacts repo; downloaded artifacts satisfy the same build requirements as zig-built artifacts so `tclsh src/make.tcl project` succeeds with downloaded artifacts in place of built ones. |
||||
@ -0,0 +1,68 @@
|
||||
# goals/ |
||||
|
||||
## Purpose |
||||
|
||||
Holds optional detail prose for goals in the root `GOALS.md` index that need more than a one-line summary to state properly. The index entry is canonical; a detail file only elaborates and never contradicts its index entry. |
||||
|
||||
## Ownership |
||||
|
||||
- Files here are owned by the goal authoring workflow described in root `GOALS.md`. |
||||
- Only the user adds or edits goals. Agents may draft detail files for user review but must not invent goals. |
||||
- The root `GOALS.md` index entry is the source of truth. If index and detail disagree, the index wins; fix the detail. |
||||
|
||||
## Local Contracts |
||||
|
||||
### When a detail file is warranted |
||||
|
||||
A goal earns a detail file when it has any of: |
||||
- Non-obvious rationale (rejected alternatives, constraints discovered, why it is not done the obvious way) |
||||
- Multi-phase plan with sub-acceptance criteria |
||||
- External references or prior art worth citing |
||||
- More than ~3 lines of prose to state properly |
||||
|
||||
Simple goals with a clear one-liner and measurable acceptance stay index-only. Do not create empty detail files for completeness. |
||||
|
||||
### Naming |
||||
|
||||
`G-<id>-<slug>.md` — e.g. `G-007-bytecompiler.md`. The `<id>` is the stable reference (taken from the `G-<id>` in the root index); the slug is human-readable and may change without breaking links as long as the ID prefix is preserved. Sortable by `ls goals/`. |
||||
|
||||
### Detail file structure (suggested) |
||||
|
||||
``` |
||||
# G-<id> <short title> |
||||
|
||||
Status: <as in index> |
||||
Scope: <as in index> |
||||
Acceptance: <as in index> |
||||
|
||||
## Context |
||||
<why this goal exists, what problem it solves> |
||||
|
||||
## Approach |
||||
<chosen direction, key design decisions> |
||||
|
||||
## Alternatives considered |
||||
- <alt A> — rejected because <reason> |
||||
- <alt B> — deferred, see G-NNN |
||||
|
||||
## Notes |
||||
<implementation notes, references, links> |
||||
``` |
||||
|
||||
### Archive |
||||
|
||||
- `goals/archive/` holds detail files for goals that have been achieved and moved to `GOALS-archive.md`. |
||||
- On archive: move `goals/G-<id>-<slug>.md` → `goals/archive/G-<id>-<slug>.md`. Do not rename the ID prefix. |
||||
- No orphan detail files: every file under `goals/` (excluding `archive/` and this `AGENTS.md`) must correspond to an `active` or `proposed` entry in the root `GOALS.md` index. |
||||
|
||||
## Work Guidance |
||||
|
||||
LF line endings (per root AGENTS.md user preferences). |
||||
|
||||
## Verification |
||||
|
||||
None — this folder is documentation only. |
||||
|
||||
## Child DOX Index |
||||
|
||||
- `archive/` — detail files for achieved/archived goals (no child AGENTS.md needed; this file's archive rules cover it) |
||||
@ -0,0 +1,46 @@
|
||||
# G-001 Pluggable console backends for non-detectable terminals |
||||
|
||||
Status: proposed |
||||
Scope: src/modules/opunk/console-999999.0a1.0.tm, src/modules/punk/console-999999.0a1.0.tm, src/modules/punk/repl-999999.0a1.0.tm, src/lib/app-punkshell/punkshell.tcl |
||||
Acceptance: a subshell started with an ssh-channel-backed and a tk-widget-backed ::opunk::Console subclass runs an interactive REPL that reads/writes through that console; size, at_eof, and can_respond are answered by the subclass overrides; the base ::opunk::Console and punk::console module are unchanged. |
||||
|
||||
## Context |
||||
|
||||
The Punk REPL talks to its user through a console object. The base `::opunk::Console` class (`src/modules/opunk/console-999999.0a1.0.tm`) represents a console as an in/out channel pair with settled capability facts, and is designed from the outset for subclassing: it is declared `-virtual` so that *"channel environments that respond like terminals but aren't platform consoles can subclass (`-extends ::opunk::Console`) and override the capability/size methods, with existing holders of console values dispatching correctly."* The `size` method comment names the example explicitly: *"Non-channel subclasses (e.g a tk text widget acting as a terminal) override this method entirely."* |
||||
|
||||
Standard terminal detection (`is_console_or_tty`, lines 151-176 of `opunk/console-999999.0a1.0.tm`) uses `twapi::GetConsoleMode` on Windows, `chan configure -inputmode`/`-mode` cross-platform, and environment hints (`MSYSTEM`, `TERM_PROGRAM`) as a fallback for mintty-without-winpty. The class comment already flags the limitation: this is a heuristic that trades false positives (piped-but-open input treated as a terminal) for false negatives, and only works for devices that either expose channel mode flags or set those environment variables. |
||||
|
||||
Terminal-like devices that don't expose any of those signals cannot be auto-detected. Two concrete cases motivate this goal: |
||||
|
||||
1. **SSH-channel consoles.** A Punk subshell reachable over an ssh connection presents a socket channel, not a platform tty. There is no `-inputmode`/`-mode`, no `twapi` handle, and no inherited `MSYSTEM`/`TERM_PROGRAM`. Standard detection returns 0, so ANSI cursor-report queries are suppressed and the REPL degrades to a default 80x24 with no capability settling. |
||||
2. **Tk-widget consoles.** A Tk text widget acting as a terminal is not a channel at all. The base `size`/`at_eof`/`is_console_or_tty`/`can_respond` methods are all channel-shaped; the class comment concedes these subclasses "override this method entirely - none of the channel/provider logic is imposed on them." |
||||
|
||||
The seam for both already exists: `punk::console::console_spec_resolve` accepts an `::opunk::Console` object value as a console spec, so a REPL can in principle be pointed at any subclass instance. What is missing is (a) the concrete subclasses and (b) the launch-time wiring that selects a non-default console for a given REPL/subshell. |
||||
|
||||
Pluggable hooks already in place that backends may use or ignore: |
||||
- `::opunk::console::waiting_chunks_arrayvar` — redirects probe-consumed bytes to a cooperative store so active readers see them. Channel-backed subclasses reusing the base `at_eof` probe benefit from this; non-channel subclasses ignore it. |
||||
- `::opunk::console::size_query_provider` — a command prefix registered by `punk::console` to give the base class access to ANSI cursor-report mechanisms. Channel-backed subclasses gain this for free when the integrating layer registers it; non-channel subclasses override `size` and never touch it. |
||||
|
||||
## Approach |
||||
|
||||
Add two reference `::opunk::Console` subclasses, each in its own module under `src/modules/opunk/` following the `-999999.0a1.0.tm` naming convention: |
||||
|
||||
1. **`opunk::console::ssh`** — wraps an ssh socket channel pair as a console. Overrides `is_console_or_tty` to return 1 (the backend is constructed knowing the channel is a terminal), `can_respond` to settle to 1, `at_eof` to check the channel's eof without the base class's pipe-probe (the channel is a known terminal, not a probed pipe), and `size` to either delegate to the registered `size_query_provider` (ANSI cursor report over the ssh channel) or query a backend-specific size if the ssh transport exposes one. Reuses `waiting_chunks_arrayvar` redirection if the probe path is exercised. |
||||
2. **`opunk::console::tk`** — wraps a Tk text widget as a console. Overrides `size` (widget dimensions), `at_eof` (a backend-specific eof marker, not a channel eof), `is_console_or_tty` (returns 1), and `can_respond` (returns 1). Does not touch `waiting_chunks_arrayvar` or `size_query_provider` — non-channel subclass. |
||||
|
||||
Then add launch-time console selection so a REPL or subshell can be started against a non-default console without hardcoding stdin/stdout. The selection seam lives at REPL init (`repl::init` in `src/modules/punk/repl-999999.0a1.0.tm`, lines ~3109-3128) where the "default" anchored console instance is currently constructed. The `-console` convention already being migrated through `punk::console` (per `src/modules/punk/AGENTS.md` "Work Guidance") is the natural extension point: accept a console spec (channel pair, instance name, or object value) at `repl::init` and resolve it via `punk::console::console_spec_resolve`, settling the resulting object's capability as appropriate for the backend. |
||||
|
||||
## Alternatives considered |
||||
|
||||
- **Auto-detect ssh/tk consoles in `is_console_or_tty`.** Rejected: the class comment already explains detection is deliberately heuristic because non-detectable devices don't expose the signals detection relies on. Adding more env-var hints or channel-shape sniffing for ssh/tk would compound the false-positive risk the comment warns about. Subclassing with explicit construction-time capability is what the `-virtual` design is for. |
||||
- **Put the ssh/tk subclasses inside `punk::console` rather than `opunk`.** Rejected: `punk::console` is the integrating layer that registers providers with the base class; putting backends there would invert the dependency direction the class comment insists on (*"the class depending on punk::console"* is explicitly avoided). Backends belong in `opunk` alongside the base class, or in their own namespaces. |
||||
- **Make `size` pluggable per-instance instead of per-class.** Rejected: the `size_query_provider` hook already covers the per-instance pluggable case for channel-based consoles. Non-channel subclasses need full method override, which is the `-virtual` dispatch path. A second per-instance hook would duplicate the existing mechanism. |
||||
- **Detect tk widgets by checking for a Tk window path.** Rejected: a Tk text widget is not a channel and shouldn't be forced through channel detection. The subclass is constructed knowing what it wraps; detection is the wrong tool. |
||||
|
||||
## Notes |
||||
|
||||
- The `-virtual` voo dispatch means existing holders of an `opunk::Console` value (e.g. `punk::console::console_spec_resolve` callers, REPL init) dispatch to the subclass methods automatically once the value carries the subclass namespace tag. No call-site changes should be needed beyond passing the new console spec at launch. |
||||
- `punk::console::ensure_object_integration` registers the size provider and redirects the probe-byte store. Channel-backed ssh subclass inherits this; tk subclass ignores it. Confirm during implementation that the tk subclass's `size` override does not accidentally fall through to the provider path. |
||||
- The base class `at_eof` does a non-blocking 1-byte probe on pipe-like channels and parks the byte in `waiting_chunks_waiting`. The ssh subclass should override `at_eof` to use `chan eof` on the ssh channel directly — probing a socket would consume a byte the protocol layer may need. |
||||
- The "first subshell asymmetry" TODO at `repl-999999.0a1.0.tm:3130-3132` is G-002's concern, not G-001's, but G-001's launch-time console selection is a prerequisite for G-002's "target a named console" acceptance criterion. Sequence G-001 before G-002. |
||||
- No persisted prior chat on this topic was found in project sessions; the motivation comes from the existing class design comments and the user's stated intent. |
||||
@ -0,0 +1,74 @@
|
||||
# G-002 Non-nested subshell with console targeting and inter-subshell comms |
||||
|
||||
Status: proposed |
||||
Scope: src/modules/punk/repl-999999.0a1.0.tm, src/modules/punk/repl/codethread-999999.0a1.0.tm |
||||
Acceptance: a parent REPL launches a subshell against a named console and continues processing its own input while the subshell runs; the parent can signal/query the running subshell; thread::send -async dispatched from within the subshell's code interp arrives at that interp (so packages like promise work when thread features aren't disabled); the "first subshell asymmetry" TODO at repl-999999.0a1.0.tm:3130 is resolved; existing synchronous `subshell punk`/`safe`/`safebase`/`punksafe` behaviour is preserved as a default mode. |
||||
|
||||
## Context |
||||
|
||||
The `::subshell` command (`src/modules/punk/repl-999999.0a1.0.tm:4147-4155`) is an alias to `::repl::interphelpers::subshell_ensemble`. The ensemble has four procs (`punk`, `safe`, `safebase`, `punksafe`, lines 3514-3560) and every one of them launches a subshell the same way: |
||||
|
||||
```tcl |
||||
set replresult [interp eval code { |
||||
package require punk::repl |
||||
repl::init -type punk |
||||
repl::start stdin |
||||
}] |
||||
``` |
||||
|
||||
Three properties follow from this shape: |
||||
|
||||
1. **Always nested.** The subshell runs in a child interpreter named `code` created earlier in `repl::init` (lines 3575-3620). The parent's call to `interp eval code { ... }` is synchronous and blocks until the child REPL exits. |
||||
2. **Always stdin.** `repl::start stdin` hardcodes the parent's stdin as the subshell's input. There is no way to point a subshell at a different console. |
||||
3. **First-subshell asymmetry.** The TODO at lines 3130-3132 notes the first subshell runs differently from subsequent nested ones, and that resolving this matters for *"control aspects of the code interp such as cpu/memory resource limits and sandboxing"* and for *"consistency for how thread calls are routed to the parent interp vs a child interp."* |
||||
|
||||
A second TODO at lines 3135-3139 is directly on-point for non-blocking communication: |
||||
|
||||
> *"investigate whether Tcl code or thread extension code is responsible for routing `thread::send -async` calls. We want to be able to control which interp receives the call. If we can't do this in pure tcl, then first investigate if we can do it purely in an alternative thread extension. Such an extension should be based closely on the existing thread extension, with minimal changes to allow us to control which interp receives the call, the intention being to see if the changes can be made unobtrusive and backwards compatible so that it has a chance of being accepted into the mainline thread extension."* |
||||
|
||||
The primitives for non-blocking inter-thread state already exist in `codethread-999999.0a1.0.tm`: `tsv::set`/`tsv::get` for shared state and `thread::cond notify`/`thread::wait` for signalling. They are used between the codethread and the repl thread but are not wired to the subshell command. The subshell is a purely nested, synchronous structure today. |
||||
|
||||
This goal depends on G-001's launch-time console selection: "target a named console" requires the REPL to accept a non-default console spec, which G-001 adds. |
||||
|
||||
## Approach |
||||
|
||||
Split the subshell launch into layers. The canonical launch API (section 0) is a prerequisite for the others — the asymmetry fix (section 4) and the non-blocking launch (section 2) both depend on a single dispatch owning `code`-interp creation. |
||||
|
||||
0. **Canonical launch API.** Make `subshell` the single public entry point for starting a REPL; demote `repl::init`+`repl::start` to internal implementation that only `subshell`'s dispatch calls. The `interp alias` mechanism — already in use at `repl-999999.0a1.0.tm:4155` (`code alias subshell ::repl::interphelpers::subshell_ensemble`) — becomes the *context signal*, not caller introspection: |
||||
- **At root:** `subshell` is a real proc (or aliased to the same dispatch in the root interp). The dispatch creates the first `code` interp, installs the `subshell` alias in it pointing back to the dispatch, and runs the REPL in `code`. |
||||
- **Inside a `code` interp:** `subshell` is the alias installed at creation time. Calling it runs the dispatch in the parent, which creates a nested `code` interp (sub-interp of the calling `code`), installs the same alias in the nested interp, and runs the REPL there. |
||||
- The caller's code is identical in both cases (`subshell punk`, `subshell safe`, etc.). The caller never introspects "am I in a `code` interp?" — the alias is the answer. Safe interps simply don't get the alias (or get a restricted one permitting only configured launch types), so they can't launch unrestricted subshells and can't ascertain their nesting level. |
||||
- Pre-configuration (`-console`, `-limits`, `-sandbox`) passes through the same API at every level; the dispatch applies it at `code`-interp creation time (see G-003). The existing `subshell punk`/`safe`/`safebase`/`punksafe` ensemble procs become type presets that set `-type` and default `-sandbox`; option keywords (`-console`, `-limits`, `-sandbox`, later `-async`) layer on top. |
||||
- **Design constraint:** the dispatch runs in the parent interp (via alias), so the parent must be responsive enough to service the alias call. In the synchronous-nested model today the parent is blocked in `interp eval` anyway. For the non-blocking model (section 2), the parent's event loop must reach the alias dispatch — this ties into the `thread::send -async` routing concern and must not be made unreachable by the non-blocking design. |
||||
|
||||
1. **Console targeting.** `repl::init` accepts a `-console` spec (channel pair, anchored instance name, or `::opunk::Console` object value) resolved via `punk::console::console_spec_resolve`. The subshell ensemble procs pass through a `-console` argument instead of hardcoding `stdin`. This is the G-001 seam extended into the subshell command. |
||||
|
||||
2. **Non-blocking launch.** Replace the synchronous `interp eval code { repl::start stdin }` with an asynchronous dispatch that returns control to the parent REPL. Two implementation directions to investigate (the user's TODO at 3135-3139 already frames this as an investigation): |
||||
- **Pure-Tcl:** keep the child interp in a separate thread and use `thread::send -async` with a `-callback` to the parent interp. The parent's REPL loop must yield to its event loop between inputs so the callback can be delivered. This requires the parent to control which interp receives the async send, which the TODO flags as the open question. |
||||
- **Thread-extension variant:** if pure-Tcl routing of `thread::send -async` to a chosen interp isn't possible, build a minimally-modified thread extension (per the TODO's mainline-acceptance criterion) and gate it behind a `package require` with fallback to the synchronous mode. |
||||
|
||||
**Hard constraint — standard thread semantics must be preserved.** Any non-blocking launch solution must keep `thread::send -async` working as it does in a plain `tclsh` session for code running inside the subshell's `code` interp. The standard Tcl `Thread` package (written in C, maintained by the Tcl core team) routes `thread::send -async` messages to the root interpreter of the target thread, not to a named sub-interp; this is documented in-code at `codethread-999999.0a1.0.tm:112-113` (*"expecting to be called from a thread::send in parent repl - ie in the toplevel interp so that the sub-interp 'code' is available"*). Packages like `promise` (vendored at `src/vendormodules/promise-1.2.0.tm`) rely on `thread::send -async` callbacks arriving at the interp that dispatched them; if the message lands at the root interp instead of the `code` interp, `promise` and similar packages break. The investigation must therefore answer: can `thread::send -async` be routed to a chosen sub-interp in pure Tcl, or does the Thread package (and potentially Tcl itself) need modification? |
||||
- If a **customised Thread package** is required: the modification is written in C, based closely on the upstream Thread extension, with minimal changes to allow controlling which interp receives the call. The existing TODO at repl:3135-3139 sets a mainline-acceptance criterion — the changes should be unobtrusive and backwards-compatible enough to have a chance of being accepted upstream. If this path is taken, create a **follow-up goal** for the fork's maintenance and upstreaming; do not let it grow inside G-002. |
||||
- If **Tcl itself** needs patching: same criterion — minimal, upstreamable. A separate follow-up goal should be created for the Tcl patch's maintenance if this path is taken. |
||||
|
||||
3. **Inter-subshell communication.** Build on the existing `tsv::`/`thread::cond` primitives already in `codethread-999999.0a1.0.tm`. A running subshell registers a well-known `tsv::` key (keyed by subshell id); the parent signals/queries via `tsv::` and `thread::cond`. The exact message vocabulary is deferred to implementation — this goal's acceptance is only that the parent can signal/query a running subshell, not a full RPC protocol. |
||||
|
||||
4. **First-subshell asymmetry.** Resolve TODO at repl:3130 by making the first subshell use the same `code`-interp launch path as subsequent ones. The asymmetry today is that the first subshell's `code` interp is created lazily/implicitly; subsequent ones reuse the pattern. The fix is a consequence of the canonical launch API (section 0): root and nested both go through the same dispatch, so there is no longer a "first subshell is special" path — the only branch in the dispatch is whether to create a top-level `code` or a nested one, which is one branch, not a separate code path. |
||||
|
||||
5. **Default mode preserved.** The existing synchronous `subshell punk`/`safe`/`safebase`/`punksafe` behaviour stays available as a default (no `-console`/`-async` flags) so existing scripts and muscle memory don't break. The new behaviour is opt-in via flags. |
||||
|
||||
## Alternatives considered |
||||
|
||||
- **Always-async, drop synchronous mode.** Rejected: backward compatibility for existing scripts and the existing `subshell punk` UX matters. The acceptance criterion explicitly preserves synchronous as the default. |
||||
- **Coroutines instead of threads for non-blocking.** Rejected for now: the subshell runs in a child *interp*, and coroutines don't cross interp boundaries. Threads are already in use (`codethread`, `%replthread%`), so extending the thread model is consistent. Coroutines could be revisited if the thread-routing TODO finds no pure-Tcl answer. |
||||
- **Separate goal for inter-subshell comms.** Considered (the split analysis proposed this as a possible G-003). Deferred: the user framed non-blocking launch and inter-subshell comms together, and the comms vocabulary depends on the launch model chosen. Splitting now risks an orphan goal whose acceptance can't be written until the launch model is fixed. Revisit after G-002's approach firms up. |
||||
- **Resolve the thread-routing TODO before writing this goal.** Rejected: the TODO is an investigation question, not a prerequisite. The goal's acceptance is satisfied by whichever implementation direction the investigation settles on; the goal itself is stable either way. |
||||
|
||||
## Notes |
||||
|
||||
- Depends on G-001's launch-time console selection. Sequence G-001 before G-002. |
||||
- The thread::send -async routing problem is the hardest open question in G-002. The standard Thread package targets the root interp of the receiving thread; the `code` sub-interp is where subshell code actually runs, so async messages miss it. `promise` (vendored at `src/vendormodules/promise-1.2.0.tm`) is the concrete canary — if `promise` works inside a subshell, the routing is correct. |
||||
- The TODO at repl:3135-3139 frames the thread-extension investigation with a mainline-acceptance criterion. If that investigation concludes a fork is needed (customised Thread package and/or Tcl patch), create a follow-up goal for the fork's maintenance and upstreaming; do not let it grow inside G-002. The fork itself is out of scope for G-002's acceptance — G-002 is satisfied by whichever routing solution is found, not by the fork existing. |
||||
- `tsv::` and `thread::cond` are already used in `codethread-999999.0a1.0.tm` (lines 203-218) for cross-thread result/status handoff. The inter-subshell comms layer should reuse the same primitives rather than introducing a new message bus. |
||||
- The "first subshell asymmetry" fix touches the same `code` interp creation path used by the synchronous mode, so it should be done first (it's the smallest piece and unblocks the others). |
||||
- No persisted prior chat on this topic was found in project sessions; the motivation comes from the existing TODOs at repl:3130-3139 and the user's stated intent. |
||||
@ -0,0 +1,48 @@
|
||||
# G-003 Configurable resource limits and sandboxing on subshell interps |
||||
|
||||
Status: proposed |
||||
Scope: src/modules/punk/repl-999999.0a1.0.tm, src/modules/punk/repl/codethread-999999.0a1.0.tm |
||||
Acceptance: a subshell can be launched with at least one resource limit (command-count via `interp limit -command`, or time via `interp limit -time`) and one sandboxing feature (e.g. `interp hide` of a command, or full safe-interp restrictions) applied to its code interp, enforceable regardless of subshell nesting depth; a subshell can be configured anywhere on the spectrum from unrestricted to fully safe via expose/hide of commands; the existing default subshell behaviour (no limits, no extra sandbox beyond the existing safe/safebase/punksafe types) is preserved when no limits are configured. |
||||
|
||||
## Context |
||||
|
||||
The subshell's user code runs in a child Tcl interpreter named `code` (created in `repl::init` at `src/modules/punk/repl-999999.0a1.0.tm:3575-3620`). The existing TODO at repl:3130-3132 is explicit that resolving the "first subshell asymmetry" is a prerequisite for this goal: |
||||
|
||||
> *"This will be important later for us to control aspects of the code interp such as cpu/memory resource limits and sandboxing."* |
||||
|
||||
Tcl provides the primitives this goal needs: |
||||
- **Resource limits:** `interp limit` supports two dimensions — command-count (`-command`, restricts the total number of Tcl commands that may be executed in the interp) and time (`-seconds`/`-millis`). Both are nestable per-interp. These are the only interp-level resource limits Tcl exposes; this goal covers exactly those two. |
||||
- **Sandboxing spectrum:** Tcl supports a continuum from unrestricted to fully sandboxed via `interp hide`/`interp expose`/`interp alias`. At one end, a plain `interp create` is unrestricted. At the other, `safe::interpCreate` produces a fully safe interpreter with hidden dangerous commands and restricted file/path access. In between, `interp hide <command>` removes specific commands, `interp expose <command>` brings them back, and `interp alias` can redirect them to restricted wrappers — so a subshell can be configured to any point on the spectrum. The existing `subshell safe`/`safebase`/`punksafe` ensemble procs (`repl-999999.0a1.0.tm:3530-3560`) already create safe interps; this goal generalises that to be configurable on the `punk`-type subshell too, and exposes the in-between sandboxing points. |
||||
|
||||
The goal is downstream of G-002 because: |
||||
1. The "first subshell asymmetry" fix (G-002) ensures all subshells — including the first — run through the same `code`-interp creation path, so limits/sandbox applied at creation time are uniform regardless of nesting depth. |
||||
2. G-002's canonical launch API design (section 0 of G-002's approach) makes the dispatch — running in the parent via `interp alias` — the single owner of `code`-interp creation. This is the property that makes limits/sandbox uniform: the dispatch applies `-limits`/`-sandbox` at creation time for every launch, at every nesting depth, because there is no other code path that creates a `code` interp. |
||||
2. Resource limits are only meaningful if the subshell actually runs code in the `code` interp; if the asymmetry persists, the first subshell's limits would need a separate code path, defeating the goal. |
||||
|
||||
The existing safe-interp variants already prove the sandboxing primitive works; this goal is about making it configurable and adding resource limits, not inventing new sandboxing technology. |
||||
|
||||
## Approach |
||||
|
||||
1. **Limit configuration surface.** Add a `-limits` option to the subshell launch (and to `repl::init` where the `code` interp is created) accepting a dict of limit specs: `{command <count>}` for command-count and `{time <seconds>}` for time limits (millisecond granularity to be decided during implementation). Apply via `interp limit code ...` immediately after `interp create code`. |
||||
|
||||
2. **Sandbox configuration surface.** Add a `-sandbox` option accepting a spec that places the subshell anywhere on the spectrum from unrestricted to fully safe. Mechanisms: `interp hide <command>` to remove specific commands, `interp expose <command>` to bring them back, `interp alias` to redirect to restricted wrappers, and `safe::interpCreate` for the fully-safe endpoint. The existing `safe`/`safebase`/`punksafe` launch types become presets that set `-sandbox` to their respective defaults, proving the general mechanism covers the existing cases. |
||||
|
||||
3. **Nesting-depth independence.** Apply limits/sandbox at `code`-interp creation time, which (after G-002's asymmetry fix) is uniform for all nesting depths. Confirm that nested subshells inherit or re-apply the configured limits rather than silently dropping them — the exact inheritance rule (do limits cascade to nested `code` interps, or must each level specify its own?) is to be decided during implementation and documented in the subshell command's help text. |
||||
|
||||
4. **Default preserved.** When neither `-limits` nor `-sandbox` is specified, the subshell launches with no limits and no extra sandbox beyond what the existing launch type already applies (e.g. `punk` type = no sandbox; `safe` type = full safe sandbox). This preserves backward compatibility. |
||||
|
||||
## Alternatives considered |
||||
|
||||
- **Always-safe subshells, drop the `punk` type.** Rejected: the `punk`-type subshell exists for trusted code that needs full Tcl access. Sandboxing should be opt-in, not mandatory. |
||||
- **OS-level limits only (process/thread, not interp).** Rejected: `interp limit` covers command-count and time at the interp level, which is finer-grained and doesn't require process boundaries. These two dimensions are what Tcl exposes; pursuing OS-level limits (e.g. memory) would be a separate goal if ever needed. |
||||
- **A new launch type (e.g. `subshell limited`) instead of options on existing types.** Rejected: options compose better than proliferating launch types. A user should be able to say `subshell punk -limits {command 100000}` without choosing a different subcommand. |
||||
- **Fold this into G-002.** Considered and rejected: resource limits + sandboxing is a distinct deliverable with its own acceptance, and the existing code comment at repl:3130-3132 already frames it as "important later" — explicitly downstream. Bundling it into G-002 would make G-002's acceptance uncheckable until this work is also done, and would grow G-002 past a one-line-safe summary. |
||||
|
||||
## Notes |
||||
|
||||
- Depends on G-002's "first subshell asymmetry" fix being complete. Sequence G-002 before G-003. |
||||
- The two interp-level resource limits Tcl exposes are command-count (`interp limit -command`) and time (`interp limit -time`). No other interp-level resource limits are in scope; if OS-level limits (e.g. memory via rlimit/job objects) are wanted later, that's a separate goal. |
||||
- The existing `safe`/`safebase`/`punksafe` ensemble procs (repl-999999.0a1.0.tm:3530-3560) are the natural testbed for the sandboxing configuration surface: they should be reimplemented as presets that set `-sandbox` to their current behaviour, proving the general mechanism covers the existing cases. |
||||
- The sandboxing spectrum is a continuum via `interp hide`/`interp expose`/`interp alias`/`safe::interpCreate`; the `-sandbox` option should expose enough of that continuum to let a user say "hide command X and Y, leave the rest" without choosing a different subcommand. |
||||
- `interp limit` with `-command` counts commands in the interp; nested interps may or may not count against the parent's limit depending on Tcl's implementation. Confirm during implementation and document in the subshell help text. |
||||
- No persisted prior chat on this topic was found in project sessions; the motivation comes from the existing TODO at repl:3130-3132 and the user's stated intent. |
||||
@ -0,0 +1,45 @@
|
||||
# G-004 No executable binaries committed to the repository |
||||
|
||||
Status: proposed |
||||
Scope: repo-wide (bin/, src/vfs/, src/vendorlib/, src/vendormodules/, src/bootsupport/) |
||||
Acceptance: a scan of the committed tree finds no executable binaries (shared libs, .exe, native .so/.dll/.dylib, bare ELF/Mach-O); any zip-based .tm modules present contain no embedded executables; the binary artifacts previously committed are retrievable via G-005 (build from source) or G-006 (pre-built download) so their removal does not break builds. |
||||
|
||||
## Context |
||||
|
||||
The ultimate aim is that the committed repository contains no executable binaries. This is a hygiene and reproducibility goal: binaries in version control are opaque to review, bloat the repo, and make the build non-reproducible from source. Today the repo violates this — binary libraries are committed in the VFS folders (`src/vfs/`), `src/vendorlib/`, `src/vendormodules/`, and `src/bootsupport/`. These are present because there is currently no automated way to rebuild or retrieve them; removing them before an alternative exists would break builds on a clean checkout. |
||||
|
||||
Zip-based `.tm` modules are a permitted exception to the "no binaries" rule because they are module archives, not executables — but a zip-based `.tm` that embeds an executable (e.g. a native shared library packaged inside the module) is not permitted. The distinction is content, not file extension. |
||||
|
||||
This goal is the integrating outcome of two enabling goals: |
||||
- **G-005** provides the build-from-source path (zig build of Tcl9 and other dependencies). |
||||
- **G-006** provides the pre-built download path with consent gating. |
||||
|
||||
Both must be in place before the committed binaries can be removed without breaking builds, so G-004 sequences after both. |
||||
|
||||
A standing repo-wide rule in root `AGENTS.md` (User Preferences) already directs agents not to commit new executable binaries, while explicitly not hassling the developer about existing vendor/vfs binaries committed pending this goal's achievement. |
||||
|
||||
## Approach |
||||
|
||||
1. **Inventory.** Identify every executable binary currently committed across `bin/`, `src/vfs/`, `src/vendorlib/`, `src/vendormodules/`, and `src/bootsupport/`. Classify each by what it is (Tcl9 shared lib, twapi, other native extension) and which goal path (G-005 build / G-006 download) will replace it. |
||||
|
||||
2. **Verify alternatives work.** Before removing any binary, confirm the corresponding G-005 build or G-006 download produces an artifact that satisfies the build. Removal without a working replacement breaks clean-checkout builds — the acceptance explicitly requires the alternative retrieval to work. |
||||
|
||||
3. **Remove.** Delete the committed binaries from the tree. Update `.gitignore` / `.fossil-settings/ignore-glob` if needed to prevent re-addition. |
||||
|
||||
4. **Add a scan to verification.** Add a scan step (script or make.tcl target) that checks the committed tree for executable binaries and fails if any are found, so regressions are caught. The scan must recognise the zip-based `.tm` exception: a `.tm` that is a zip is allowed; a `.tm` (or any zip) containing an embedded executable is not. |
||||
|
||||
5. **DOX update.** Once achieved, the AGENTS.md "no binaries" user-preference note can be simplified from "agents must not; existing violations pending" to "agents must not; scan enforced" — but the rule itself stays. |
||||
|
||||
## Alternatives considered |
||||
|
||||
- **Delete binaries now, document manual fetch.** Rejected: no automated retrieval, so every contributor hand-fetches. The intended system (G-005/G-006) exists precisely to avoid this. |
||||
- **Allow committed binaries permanently.** Rejected: this is the explicit ultimate aim the user stated. Not a real alternative. |
||||
- **Permit zip-based .tm with embedded executables.** Rejected: the user's wording is explicit — "those zip-based .tm modules that contain executables should also not be directly checked in." The content matters, not the extension. |
||||
- **Fold G-005 and G-006 into this goal.** Rejected: the build system and download system are each large bodies of work with their own acceptance, and they're independently useful before all binaries are removed. Bundling would make G-004 uncheckable until two large systems are built, and would grow past a one-line-safe summary. |
||||
|
||||
## Notes |
||||
|
||||
- Depends on G-005 and G-006. Sequence G-005 ∥ G-006, then G-004. |
||||
- The scan in step 4 should distinguish "executable binary" from "zip archive" and from "zip archive containing an executable." A naive extension-based check is insufficient; content inspection (file magic, zip listing) is required. |
||||
- The AGENTS.md user-preference rule is the transition guard: it stops agents from adding new binaries during the G-005/G-006 development period while the developer's existing vendor/vfs binary commits are known and intentional. |
||||
- No persisted prior chat on this topic was found in project sessions; the motivation is the user's stated ultimate aim. |
||||
@ -0,0 +1,45 @@
|
||||
# G-005 Zig-based build infrastructure for binary dependencies from source |
||||
|
||||
Status: proposed |
||||
Scope: src/runtime/, build.zig / build.zig.zon (new), src/make.tcl integration |
||||
Acceptance: running the zig build produces the binary artifacts the repo previously committed (at minimum: Tcl9 library for one target platform); existing Tcl9-zig experiments brought into the project; `tclsh src/make.tcl` integrates with the zig build so a normal project build retrieves/builds binaries via zig when not present; no binary artifacts need to be committed for the build to succeed on a clean checkout with the zig toolchain available. |
||||
|
||||
## Context |
||||
|
||||
The repo currently commits binary dependencies (Tcl9 shared library, twapi, other native extensions) in VFS, vendorlib, vendormodules, and bootsupport folders. G-004's aim is to remove these, but that requires an automated way to rebuild them from source. This goal is that mechanism. |
||||
|
||||
The intended build infrastructure is the **zig language and build system**, chosen for its cross-platform compilation support. Experiments building Tcl9 with zig (instead of the standard autoconf/make Tcl build) have already been performed outside this project and will be brought in for expansion. Zig's `build.zig` provides a cross-platform build graph that can compile C code (Tcl is C) for multiple targets from a single toolchain, which is why it was chosen over the standard Tcl build system for this purpose. |
||||
|
||||
The binary dependencies to cover include at minimum: |
||||
- Tcl9 shared library (the core that the Punk shell runs on) |
||||
- Native Tcl extensions currently vendored (e.g. twapi on Windows, tcllibc) |
||||
- Any other native components the build currently expects to find committed |
||||
|
||||
This goal is the primary mechanism for G-004's outcome. It is independent of and parallel to G-006 (pre-built download); the two provide alternative paths to the same artifacts. |
||||
|
||||
## Approach |
||||
|
||||
1. **Bring in the Tcl9-zig experiment.** Import the existing out-of-project Tcl9-zig build experiment into the repo (likely under `src/runtime/` or a new top-level build directory). Get it building Tcl9 for at least one target platform. |
||||
|
||||
2. **Expand to other dependencies.** Extend the zig build to cover the other native binary dependencies currently committed (twapi, tcllibc, etc.), one at a time. Each dependency gets a zig build target that produces the same artifact shape the repo currently commits. |
||||
|
||||
3. **Integrate with make.tcl.** `tclsh src/make.tcl` is the existing build entry point. Wire it so that when a required binary artifact is not present, it invokes the zig build to produce it (or, per G-006, offers the download path). The integration must not require a separate manual zig step for a normal `make.tcl project` build when the zig toolchain is available. |
||||
|
||||
4. **Document the zig toolchain prerequisite.** The zig toolchain is a build-time prerequisite for this path. Document how to obtain it (install instructions, version pinning). Consider auto-detection with an actionable message when zig is absent, and defer to G-006's download path or an error depending on what the user has configured. |
||||
|
||||
5. **Cross-platform targets.** The project's primary target is Windows (`win32-x86_64`); Linux, macOS, FreeBSD are secondary. The zig build should support at least the primary target first, with the cross-platform capability used to add the others. |
||||
|
||||
## Alternatives considered |
||||
|
||||
- **Standard Tcl build (autoconf/make).** Rejected for this purpose: the standard Tcl build system is platform-specific and doesn't provide the cross-platform single-toolchain compilation that zig does. The zig experiment exists precisely because the standard build was judged insufficient for this project's cross-platform needs. |
||||
- **Keep committed binaries, no build infrastructure.** Rejected: this is the explicit problem G-004 exists to solve. |
||||
- **CMake or other build systems.** Not chosen; the user specified zig, and the Tcl9-zig experiment already exists. Switching systems would discard the existing experiment. |
||||
- **Fold the download path (G-006) into this goal.** Rejected: the consent-gating requirement on downloads is a distinct behavioural concern with its own acceptance, and the two paths are independently useful (a user with zig builds; a user without zig downloads). |
||||
|
||||
## Notes |
||||
|
||||
- Parallel to G-006. Both produce the same artifact set; G-004's acceptance allows either path. |
||||
- The zig build produces artifacts that `make.tcl` expects to find in specific locations (VFS, vendor folders). The integration must place outputs where the existing build looks for them, or update the existing build to look in the zig output locations. |
||||
- The Tcl9-zig experiment is the seed; expect it to need expansion beyond Tcl9 to cover the full dependency set. Don't underestimate this — the experiment covers one component; the goal covers all committed binaries. |
||||
- If the zig build for a particular dependency proves infeasible, G-006's download path is the fallback for that dependency, and the goal's acceptance is satisfied by the dependencies it does cover plus download for the rest. Document any such deferral. |
||||
- No persisted prior chat on this topic was found in project sessions; the motivation is the user's stated intent and the existing Tcl9-zig experiment. |
||||
@ -0,0 +1,45 @@
|
||||
# G-006 Optional pre-built binary artifact download with consent gating |
||||
|
||||
Status: proposed |
||||
Scope: src/runtime/, src/make.tcl integration, user-config (consent flags) |
||||
Acceptance: a download mechanism fetches binary artifacts (the same set the zig build produces) from a configured source on demand; by default the download is gated behind explicit user consent (a config flag or interactive prompt) and does not occur silently; a user-configured source URL overrides the default binary-artifacts repo; downloaded artifacts satisfy the same build requirements as zig-built artifacts so `tclsh src/make.tcl project` succeeds with downloaded artifacts in place of built ones. |
||||
|
||||
## Context |
||||
|
||||
Not every user of Punkshell will have the zig toolchain installed or want to build binary dependencies from source. This goal provides the alternative: download pre-built artifacts. It is the complementary mechanism to G-005 (zig build); both produce the same artifact set that G-004 requires in place of committed binaries. |
||||
|
||||
Two source categories: |
||||
1. **Default:** a separate, related binary-artifacts repository (sibling to this source repo, not part of it) holding pre-built artifacts for supported platforms. |
||||
2. **User-configured:** a user-supplied source URL (internal mirror, local file server, etc.) that overrides the default. |
||||
|
||||
The critical behavioural requirement is **consent gating by default**: the download must not happen silently. A first-time build that silently reaches out to a remote source would be surprising and a network-exfiltration concern. The user must explicitly opt in — via a config flag set before the build, or an interactive prompt at build time — before any download occurs. Once consent is given (e.g. a config flag persisted), subsequent builds for the same artifact set don't re-prompt. |
||||
|
||||
This goal is parallel to and independent of G-005. A user with zig uses G-005; a user without zig (or choosing not to build) uses G-006. Either path satisfies G-004's retrieval requirement. |
||||
|
||||
## Approach |
||||
|
||||
1. **Download mechanism.** Implement a downloader (likely in `src/runtime/` or integrated into `make.tcl`) that fetches a named artifact from a configured source URL to the location the build expects. Verify checksums/signatures where the artifact source provides them. |
||||
|
||||
2. **Consent gate.** Before any download, check a consent flag in user config. If unset, either prompt interactively (when the build is interactive) or fail with an actionable message telling the user how to set the flag (when non-interactive). Once set, record the consent so subsequent builds don't re-prompt. The gate must not be bypassable by accident — a silent download on first build is a failure of this goal's acceptance. |
||||
|
||||
3. **Source configuration.** A config value for the artifact source URL, defaulting to the separate binary-artifacts repo. A user-configured value overrides the default. Document the config key and the expected source format (URL layout, artifact naming convention). |
||||
|
||||
4. **Integration with make.tcl.** When `make.tcl` needs a binary artifact that isn't present and the zig build path (G-005) is not available or not chosen, the download path is offered/followed per the consent gate. The integration must make `tclsh src/make.tcl project` succeed with downloaded artifacts in place of built ones — the artifacts are interchangeable from the build's perspective. |
||||
|
||||
5. **Artifact equivalence.** Downloaded artifacts must satisfy the same build requirements as zig-built ones (same file shape, same location, same version). If a downloaded artifact differs in a way the build cares about (e.g. different Tcl9 patchlevel), the build must detect and reject it rather than producing a subtly broken system. |
||||
|
||||
## Alternatives considered |
||||
|
||||
- **Always download silently on first build.** Rejected: the user's requirement is explicit consent gating. Silent network access on first build is a privacy/security concern and surprising behaviour. |
||||
- **Download only, no build path (drop G-005).** Rejected: the user wants both paths. Build-from-source (G-005) is the reproducible/auditable path; download is the convenience path. |
||||
- **Interactive prompt only, no config flag.** Rejected: non-interactive builds (CI, scripts) would have no way to consent. Both a config flag (for non-interactive) and a prompt (for interactive) are needed. |
||||
- **Bundle the binary-artifacts repo into this repo.** Rejected: that would defeat G-004's aim — the binary artifacts would still be in version control, just in a sub-repo. The artifacts repo is separate by design. |
||||
|
||||
## Notes |
||||
|
||||
- Parallel to G-005. Both feed G-004. |
||||
- The separate binary-artifacts repository is a sibling to this source repo, not a subdirectory or submodule. Its maintenance is out of scope for this goal; this goal only consumes from it. |
||||
- The consent gate is the defining behavioural requirement of this goal. A download mechanism without the gate fails the acceptance even if the artifacts arrive correctly. |
||||
- Consider version-pinning: downloaded artifacts should match the versions the build expects, not "latest". The source URL scheme should encode version so a user gets a consistent artifact set. |
||||
- If a user configures a source that serves a malicious/tampered artifact, checksum/signature verification is the mitigation. Whether the default binary-artifacts repo signs artifacts is a decision for that repo; this goal's acceptance is satisfied by the download + consent mechanism, with verification as a hardening step. |
||||
- No persisted prior chat on this topic was found in project sessions; the motivation is the user's stated intent. |
||||
@ -0,0 +1 @@
|
||||
# Placeholder so the archive directory is tracked before any files are moved into it. |
||||
Loading…
Reference in new issue