Browse Source
Index entries in GOALS.md plus detail files. G-004 targets a binary-free committed tree, achievable once G-005 (zig-based build of binary deps from source) or G-006 (consent-gated download of pre-built artifacts) provides the replacement supply of the previously committed binaries. Assisted-by: harness=claude; primary-model=claude-fable-5; api-location=anthropic.commaster
4 changed files with 153 additions and 0 deletions
@ -0,0 +1,45 @@
|
||||
# G-004 No executable binaries committed to the repository |
||||
|
||||
Status: proposed |
||||
Scope: repo-wide (bin/, src/vfs/, src/vendorlib/, src/vendormodules/, src/bootsupport/) |
||||
Acceptance: a scan of the committed tree finds no executable binaries (shared libs, .exe, native .so/.dll/.dylib, bare ELF/Mach-O); any zip-based .tm modules present contain no embedded executables; the binary artifacts previously committed are retrievable via G-005 (build from source) or G-006 (pre-built download) so their removal does not break builds. |
||||
|
||||
## Context |
||||
|
||||
The ultimate aim is that the committed repository contains no executable binaries. This is a hygiene and reproducibility goal: binaries in version control are opaque to review, bloat the repo, and make the build non-reproducible from source. Today the repo violates this — binary libraries are committed in the VFS folders (`src/vfs/`), `src/vendorlib/`, `src/vendormodules/`, and `src/bootsupport/`. These are present because there is currently no automated way to rebuild or retrieve them; removing them before an alternative exists would break builds on a clean checkout. |
||||
|
||||
Zip-based `.tm` modules are a permitted exception to the "no binaries" rule because they are module archives, not executables — but a zip-based `.tm` that embeds an executable (e.g. a native shared library packaged inside the module) is not permitted. The distinction is content, not file extension. |
||||
|
||||
This goal is the integrating outcome of two enabling goals: |
||||
- **G-005** provides the build-from-source path (zig build of Tcl9 and other dependencies). |
||||
- **G-006** provides the pre-built download path with consent gating. |
||||
|
||||
Both must be in place before the committed binaries can be removed without breaking builds, so G-004 sequences after both. |
||||
|
||||
A standing repo-wide rule in root `AGENTS.md` (User Preferences) already directs agents not to commit new executable binaries, while explicitly not hassling the developer about existing vendor/vfs binaries committed pending this goal's achievement. |
||||
|
||||
## Approach |
||||
|
||||
1. **Inventory.** Identify every executable binary currently committed across `bin/`, `src/vfs/`, `src/vendorlib/`, `src/vendormodules/`, and `src/bootsupport/`. Classify each by what it is (Tcl9 shared lib, twapi, other native extension) and which goal path (G-005 build / G-006 download) will replace it. |
||||
|
||||
2. **Verify alternatives work.** Before removing any binary, confirm the corresponding G-005 build or G-006 download produces an artifact that satisfies the build. Removal without a working replacement breaks clean-checkout builds — the acceptance explicitly requires the alternative retrieval to work. |
||||
|
||||
3. **Remove.** Delete the committed binaries from the tree. Update `.gitignore` / `.fossil-settings/ignore-glob` if needed to prevent re-addition. |
||||
|
||||
4. **Add a scan to verification.** Add a scan step (script or make.tcl target) that checks the committed tree for executable binaries and fails if any are found, so regressions are caught. The scan must recognise the zip-based `.tm` exception: a `.tm` that is a zip is allowed; a `.tm` (or any zip) containing an embedded executable is not. |
||||
|
||||
5. **DOX update.** Once achieved, the AGENTS.md "no binaries" user-preference note can be simplified from "agents must not; existing violations pending" to "agents must not; scan enforced" — but the rule itself stays. |
||||
|
||||
## Alternatives considered |
||||
|
||||
- **Delete binaries now, document manual fetch.** Rejected: no automated retrieval, so every contributor hand-fetches. The intended system (G-005/G-006) exists precisely to avoid this. |
||||
- **Allow committed binaries permanently.** Rejected: this is the explicit ultimate aim the user stated. Not a real alternative. |
||||
- **Permit zip-based .tm with embedded executables.** Rejected: the user's wording is explicit — "those zip-based .tm modules that contain executables should also not be directly checked in." The content matters, not the extension. |
||||
- **Fold G-005 and G-006 into this goal.** Rejected: the build system and download system are each large bodies of work with their own acceptance, and they're independently useful before all binaries are removed. Bundling would make G-004 uncheckable until two large systems are built, and would grow past a one-line-safe summary. |
||||
|
||||
## Notes |
||||
|
||||
- Depends on G-005 and G-006. Sequence G-005 ∥ G-006, then G-004. |
||||
- The scan in step 4 should distinguish "executable binary" from "zip archive" and from "zip archive containing an executable." A naive extension-based check is insufficient; content inspection (file magic, zip listing) is required. |
||||
- The AGENTS.md user-preference rule is the transition guard: it stops agents from adding new binaries during the G-005/G-006 development period while the developer's existing vendor/vfs binary commits are known and intentional. |
||||
- No persisted prior chat on this topic was found in project sessions; the motivation is the user's stated ultimate aim. |
||||
@ -0,0 +1,45 @@
|
||||
# G-005 Zig-based build infrastructure for binary dependencies from source |
||||
|
||||
Status: proposed |
||||
Scope: src/runtime/, build.zig / build.zig.zon (new), src/make.tcl integration |
||||
Acceptance: running the zig build produces the binary artifacts the repo previously committed (at minimum: Tcl9 library for one target platform); existing Tcl9-zig experiments brought into the project; `tclsh src/make.tcl` integrates with the zig build so a normal project build retrieves/builds binaries via zig when not present; no binary artifacts need to be committed for the build to succeed on a clean checkout with the zig toolchain available. |
||||
|
||||
## Context |
||||
|
||||
The repo currently commits binary dependencies (Tcl9 shared library, twapi, other native extensions) in VFS, vendorlib, vendormodules, and bootsupport folders. G-004's aim is to remove these, but that requires an automated way to rebuild them from source. This goal is that mechanism. |
||||
|
||||
The intended build infrastructure is the **zig language and build system**, chosen for its cross-platform compilation support. Experiments building Tcl9 with zig (instead of the standard autoconf/make Tcl build) have already been performed outside this project and will be brought in for expansion. Zig's `build.zig` provides a cross-platform build graph that can compile C code (Tcl is C) for multiple targets from a single toolchain, which is why it was chosen over the standard Tcl build system for this purpose. |
||||
|
||||
The binary dependencies to cover include at minimum: |
||||
- Tcl9 shared library (the core that the Punk shell runs on) |
||||
- Native Tcl extensions currently vendored (e.g. twapi on Windows, tcllibc) |
||||
- Any other native components the build currently expects to find committed |
||||
|
||||
This goal is the primary mechanism for G-004's outcome. It is independent of and parallel to G-006 (pre-built download); the two provide alternative paths to the same artifacts. |
||||
|
||||
## Approach |
||||
|
||||
1. **Bring in the Tcl9-zig experiment.** Import the existing out-of-project Tcl9-zig build experiment into the repo (likely under `src/runtime/` or a new top-level build directory). Get it building Tcl9 for at least one target platform. |
||||
|
||||
2. **Expand to other dependencies.** Extend the zig build to cover the other native binary dependencies currently committed (twapi, tcllibc, etc.), one at a time. Each dependency gets a zig build target that produces the same artifact shape the repo currently commits. |
||||
|
||||
3. **Integrate with make.tcl.** `tclsh src/make.tcl` is the existing build entry point. Wire it so that when a required binary artifact is not present, it invokes the zig build to produce it (or, per G-006, offers the download path). The integration must not require a separate manual zig step for a normal `make.tcl project` build when the zig toolchain is available. |
||||
|
||||
4. **Document the zig toolchain prerequisite.** The zig toolchain is a build-time prerequisite for this path. Document how to obtain it (install instructions, version pinning). Consider auto-detection with an actionable message when zig is absent, and defer to G-006's download path or an error depending on what the user has configured. |
||||
|
||||
5. **Cross-platform targets.** The project's primary target is Windows (`win32-x86_64`); Linux, macOS, FreeBSD are secondary. The zig build should support at least the primary target first, with the cross-platform capability used to add the others. |
||||
|
||||
## Alternatives considered |
||||
|
||||
- **Standard Tcl build (autoconf/make).** Rejected for this purpose: the standard Tcl build system is platform-specific and doesn't provide the cross-platform single-toolchain compilation that zig does. The zig experiment exists precisely because the standard build was judged insufficient for this project's cross-platform needs. |
||||
- **Keep committed binaries, no build infrastructure.** Rejected: this is the explicit problem G-004 exists to solve. |
||||
- **CMake or other build systems.** Not chosen; the user specified zig, and the Tcl9-zig experiment already exists. Switching systems would discard the existing experiment. |
||||
- **Fold the download path (G-006) into this goal.** Rejected: the consent-gating requirement on downloads is a distinct behavioural concern with its own acceptance, and the two paths are independently useful (a user with zig builds; a user without zig downloads). |
||||
|
||||
## Notes |
||||
|
||||
- Parallel to G-006. Both produce the same artifact set; G-004's acceptance allows either path. |
||||
- The zig build produces artifacts that `make.tcl` expects to find in specific locations (VFS, vendor folders). The integration must place outputs where the existing build looks for them, or update the existing build to look in the zig output locations. |
||||
- The Tcl9-zig experiment is the seed; expect it to need expansion beyond Tcl9 to cover the full dependency set. Don't underestimate this — the experiment covers one component; the goal covers all committed binaries. |
||||
- If the zig build for a particular dependency proves infeasible, G-006's download path is the fallback for that dependency, and the goal's acceptance is satisfied by the dependencies it does cover plus download for the rest. Document any such deferral. |
||||
- No persisted prior chat on this topic was found in project sessions; the motivation is the user's stated intent and the existing Tcl9-zig experiment. |
||||
@ -0,0 +1,45 @@
|
||||
# G-006 Optional pre-built binary artifact download with consent gating |
||||
|
||||
Status: proposed |
||||
Scope: src/runtime/, src/make.tcl integration, user-config (consent flags) |
||||
Acceptance: a download mechanism fetches binary artifacts (the same set the zig build produces) from a configured source on demand; by default the download is gated behind explicit user consent (a config flag or interactive prompt) and does not occur silently; a user-configured source URL overrides the default binary-artifacts repo; downloaded artifacts satisfy the same build requirements as zig-built artifacts so `tclsh src/make.tcl project` succeeds with downloaded artifacts in place of built ones. |
||||
|
||||
## Context |
||||
|
||||
Not every user of Punkshell will have the zig toolchain installed or want to build binary dependencies from source. This goal provides the alternative: download pre-built artifacts. It is the complementary mechanism to G-005 (zig build); both produce the same artifact set that G-004 requires in place of committed binaries. |
||||
|
||||
Two source categories: |
||||
1. **Default:** a separate, related binary-artifacts repository (sibling to this source repo, not part of it) holding pre-built artifacts for supported platforms. |
||||
2. **User-configured:** a user-supplied source URL (internal mirror, local file server, etc.) that overrides the default. |
||||
|
||||
The critical behavioural requirement is **consent gating by default**: the download must not happen silently. A first-time build that silently reaches out to a remote source would be surprising and a network-exfiltration concern. The user must explicitly opt in — via a config flag set before the build, or an interactive prompt at build time — before any download occurs. Once consent is given (e.g. a config flag persisted), subsequent builds for the same artifact set don't re-prompt. |
||||
|
||||
This goal is parallel to and independent of G-005. A user with zig uses G-005; a user without zig (or choosing not to build) uses G-006. Either path satisfies G-004's retrieval requirement. |
||||
|
||||
## Approach |
||||
|
||||
1. **Download mechanism.** Implement a downloader (likely in `src/runtime/` or integrated into `make.tcl`) that fetches a named artifact from a configured source URL to the location the build expects. Verify checksums/signatures where the artifact source provides them. |
||||
|
||||
2. **Consent gate.** Before any download, check a consent flag in user config. If unset, either prompt interactively (when the build is interactive) or fail with an actionable message telling the user how to set the flag (when non-interactive). Once set, record the consent so subsequent builds don't re-prompt. The gate must not be bypassable by accident — a silent download on first build is a failure of this goal's acceptance. |
||||
|
||||
3. **Source configuration.** A config value for the artifact source URL, defaulting to the separate binary-artifacts repo. A user-configured value overrides the default. Document the config key and the expected source format (URL layout, artifact naming convention). |
||||
|
||||
4. **Integration with make.tcl.** When `make.tcl` needs a binary artifact that isn't present and the zig build path (G-005) is not available or not chosen, the download path is offered/followed per the consent gate. The integration must make `tclsh src/make.tcl project` succeed with downloaded artifacts in place of built ones — the artifacts are interchangeable from the build's perspective. |
||||
|
||||
5. **Artifact equivalence.** Downloaded artifacts must satisfy the same build requirements as zig-built ones (same file shape, same location, same version). If a downloaded artifact differs in a way the build cares about (e.g. different Tcl9 patchlevel), the build must detect and reject it rather than producing a subtly broken system. |
||||
|
||||
## Alternatives considered |
||||
|
||||
- **Always download silently on first build.** Rejected: the user's requirement is explicit consent gating. Silent network access on first build is a privacy/security concern and surprising behaviour. |
||||
- **Download only, no build path (drop G-005).** Rejected: the user wants both paths. Build-from-source (G-005) is the reproducible/auditable path; download is the convenience path. |
||||
- **Interactive prompt only, no config flag.** Rejected: non-interactive builds (CI, scripts) would have no way to consent. Both a config flag (for non-interactive) and a prompt (for interactive) are needed. |
||||
- **Bundle the binary-artifacts repo into this repo.** Rejected: that would defeat G-004's aim — the binary artifacts would still be in version control, just in a sub-repo. The artifacts repo is separate by design. |
||||
|
||||
## Notes |
||||
|
||||
- Parallel to G-005. Both feed G-004. |
||||
- The separate binary-artifacts repository is a sibling to this source repo, not a subdirectory or submodule. Its maintenance is out of scope for this goal; this goal only consumes from it. |
||||
- The consent gate is the defining behavioural requirement of this goal. A download mechanism without the gate fails the acceptance even if the artifacts arrive correctly. |
||||
- Consider version-pinning: downloaded artifacts should match the versions the build expects, not "latest". The source URL scheme should encode version so a user gets a consistent artifact set. |
||||
- If a user configures a source that serves a malicious/tampered artifact, checksum/signature verification is the mitigation. Whether the default binary-artifacts repo signs artifacts is a decision for that repo; this goal's acceptance is satisfied by the download + consent mechanism, with verification as a hardening step. |
||||
- No persisted prior chat on this topic was found in project sessions; the motivation is the user's stated intent. |
||||
Loading…
Reference in new issue