Browse Source

G-142 drafted (user-approved): punkbin curated listing manifests - index entry, detail file, G-141 back-pointer

Per-tier listing manifests with named groups (current/lts/vulnerable/archive) separating the listing surface from the sha1sums integrity ledger; activation assessed as immediately possible (no goal dependencies; G-141 libfetch surface coordination only).

Assisted-by: harness=opencode; primary-model=huggingface/moonshotai/Kimi-K3; api-location=unknown
master
Julian Noble 2 days ago
parent
commit
e53087c6d0
  1. 4
      GOALS.md
  2. 4
      goals/G-141-libfetch-target-selection.md
  3. 45
      goals/G-142-punkbin-listing-manifests.md

4
GOALS.md

@ -406,3 +406,7 @@ Detail: goals/G-137-kit-version-resources.md
Scope: src/make.tcl (libfetch target-set derivation + narrow/widen options; VFSLIBS missing-source hint; PACKAGES phase as the host-need input); src/runtime/libpackages.toml (declaration surface as consumed); src/runtime/vendorlib_vfs.toml (source_root="packages" entries as the kit-consumption input - format coordinated with G-115/G-127); src/tests/shell/testsuites/punkexe/maketcllibfetch.test (characterization: derivation, narrowing, widening); bin/AGENTS.md + src/runtime/AGENTS.md (derivation rule documentation)
Detail: goals/G-141-libfetch-target-selection.md
### G-142 [proposed] Punkbin curated listing manifests: per-tier artifact grouping with named groups
Scope: punkbin artifact repo (external c:/repo/jn/punkbin - layout contract, src/build_sha1sums.tcl, per-tier listing manifests); src/scriptapps/bin/ (punk-runtime payloads); bin/punk-runtime.cmd (regenerated output) + bin/AGENTS.md; src/tests/shell/testsuites/binscripts/; src/make.tcl (libfetch coordination)
Detail: goals/G-142-punkbin-listing-manifests.md

4
goals/G-141-libfetch-target-selection.md

@ -99,6 +99,10 @@ has no equivalent of "materialize what this checkout needs".
to consumption.
- Related: G-027 - derived projects receive libfetch via layout sync and
inherit selection unchanged.
- Related: G-142 - punkbin per-tier listing manifests; its libfetch-side
"declared artifact not in the server's current listing" warning coordinates
with this goal's libfetch surface (whichever lands second adopts the
other's vocabulary).
- Overlap survey 2026-07-31 (drafting): goals_xref paths over src/make.tcl,
src/runtime/libpackages.toml, src/runtime/vendorlib_vfs.toml, bin/packages,
src/tests/shell/testsuites/punkexe - strongest live intersections G-127 and

45
goals/G-142-punkbin-listing-manifests.md

@ -0,0 +1,45 @@
# G-142 Punkbin curated listing manifests: per-tier artifact grouping with named groups
Status: proposed
Scope: punkbin artifact repo (external c:/repo/jn/punkbin - layout contract, src/build_sha1sums.tcl, per-tier listing manifests); src/scriptapps/bin/ (punk-runtime payloads); bin/punk-runtime.cmd (regenerated output) + bin/AGENTS.md; src/tests/shell/testsuites/binscripts/; src/make.tcl (libfetch coordination)
Goal: punkbin artifact tiers (runtime platform folders, tools/, lib/<target>/) gain a curated per-tier listing manifest in the defaults.txt idiom (hand-curated, generator-validated, mirror-curable) that separates the listing surface from the integrity ledger: sha1sums.txt stays the complete integrity authority covering every retained immutable artifact, while a per-tier manifest names the listed subset in named groups with tool-understood semantics (current, lts, vulnerable carve-outs, operator-invented groups) - so an operator can retain old immutable artifacts indefinitely without punkshell-side listing tools drowning the current set in history, while every retained artifact stays fetchable and sha1-verifiable by explicit name.
Acceptance: (1) the punkbin layout contract (AGENTS.md + README) documents the per-tier listing manifest: file format (line-based, '#' comments/blanks ignored, two-column '<name> <group>'), the group vocabulary with reserved-group semantics (current = default listing; lts = listed in own section; vulnerable = hidden from default listing with a fetch-time warning; archive = hidden, fetch-silent; unknown groups listed as own sections; absent-from-manifest = retained-unlisted, fetchable by explicit name) and the mirror-curation posture; (2) build_sha1sums.tcl validates each manifest against its tier's sha1sums.txt (listed names must exist there) and reports per-tier listed/hidden/unlisted-retained counts; (3) punk-runtime list -remote renders the curated view by default (grouped sections; vulnerable + archive groups hidden) with an -all escape to the full sha1sums view, byte-agreeing across the ps1/bash payloads; (4) fetch of a vulnerable-group artifact succeeds with a warning naming the group; fetch of any retained unlisted artifact still verifies against the full sha1sums.txt; (5) binscripts characterization tests pin the listing/filtering/warning behaviours against the httpfixture server in both payloads; (6) libfetch warns when a declared artifact is absent from the server's current listing (surface coordinated with G-141).
## Context
The punkbin artifact repo's tiers are deliberately flat below their <arch> levels (runtime platform folders, tools/, lib/<target>/), and artifacts are immutable -r<N> names retained indefinitely by operator policy. Today sha1sums.txt is both the integrity authority AND the de-facto listing source: punk-runtime list -remote renders every row including remote-only entries, so retaining old revisions indefinitely grows the listing monotonically while the signal (what should I fetch?) drowns in history. defaults.txt already proves the curated-manifest idiom in this layout (hand-curated release decision, generator-validated, mirror-curable) but names only ONE artifact per platform - it cannot express "these N artifacts are the current set", nor named groups.
User direction shaping the design (2026-07-31): grouping is a first-class need - anticipated groups include LTS tracks and carve-outs for releases with known vulnerabilities (still servable by explicit name, excluded from the main listing, warning on fetch); there are NO punkbin mirrors yet and the operator is currently free to rearrange the repo; backwards compatibility is NOT a requirement for now (single corporate group, JN-controlled - it becomes a concern only once punkshell/punkbin have external consumers).
## Approach
- Per-tier curated manifest (filename settled in the work - listing.txt the working name) beside each tier's sha1sums.txt, in the defaults.txt idiom: line-based, '#' comments/blanks ignored, two-column '<name> <group>'; hand-curated as a release decision in the publication change-set, never generated; mirrors may curate their own (defaults.txt precedent).
- Group vocabulary is OPEN with reserved-group semantics understood by tools: current = the default listing; lts = listed in its own section (a still-recommended track); vulnerable = hidden from the default listing, fetchable by explicit name with a fetch-time warning naming the group (the known-vulnerability carve-out - still servable, never silently listed); archive = hidden, fetch-silent (deliberately-retained history marker); operator-invented groups are listed as their own sections; names absent from the manifest are retained-unlisted - fetchable and sha1-verifiable by explicit name, never listed (the zero-curation bucket for bulk history).
- sha1sums.txt stays the COMPLETE integrity authority: every retained artifact, listed or not, stays fetchable by explicit name and sha1-verifiable. The manifest subsets listing, never integrity.
- build_sha1sums.tcl validates each manifest against its tier's sha1sums.txt (every listed name must exist there - the same drift discipline as defaults.txt) and reports per-tier listed/hidden/unlisted-retained counts as an audit surface.
- punk-runtime list -remote renders the curated view by default (grouped sections; vulnerable + archive groups hidden) with an -all escape to the full sha1sums view; fetch of a vulnerable-group artifact succeeds with a warning naming the group; fetch of unlisted artifacts verifies against the full sha1sums.txt unchanged. Both payloads agree byte-for-byte (the established ps1/bash parity discipline + re-wrap roundtrip test).
- libfetch warns when a declared artifact is absent from the server's current listing (drift signal for the declaring project; surface coordinated with G-141).
- Backwards compatibility is deliberately NOT a design constraint (user direction 2026-07-31). Tolerant defaults are kept only where free: an absent manifest means list-everything - the natural uncurated-server behaviour, which also serves future third-party mirrors using the layout.
## Alternatives considered
- Physical archive/ subfolder per tier - reconsidered given the recorded freedom to rearrange punkbin (no mirrors yet), still rejected: an artifact's URL should be as immutable as its bytes (locator stability for pinned references), and fetch-by-name stays a single fixed path per tier instead of a main-then-archive probe chain. Revisit only if actual pruning (not listing) becomes the need.
- Display-side grouping only (tools collapse old -r<N> revisions by parsed name series) - deferred complement, rejected as the mechanism: no operator curation (cannot express LTS tracks or vulnerability carve-outs) and sha1sums.txt keeps growing as the transfer unit.
- Sharded integrity (sha1sums.txt + sha1sums-archive.txt) - rejected: splits the integrity authority and forces every consumer to answer "which file do I verify against"; the manifest subsets listing, never integrity.
- Single root-level manifest covering all tiers (defaults.txt's own shape) - rejected: per-tier files sit beside the sha1sums.txt they subset (locality), let mirrors curate per tier, and keep each tier's curation surface proportional to its churn.
- Single-column names with '#' comment group headers - rejected: two-column '<name> <group>' is machine-renderable (grouped sections, reserved-group semantics) at the same curation cost; comment headers are human-only grouping.
## Notes
- Related: G-123 (achieved 2026-07-31 - see goals/archive/G-123-thirdparty-runtime-tiers.md) - runtime-tier twin; this goal's manifest sits beside its retroactive sidecars and schema-v2 records.
- Related: G-138/G-139/G-140 (archived) - the lib tier's emission/consumption; the manifest mechanism serves lib/<target>/ identically to runtime tiers, and libfetch is a consumer.
- Related: G-141 - libfetch target selection; the libfetch-side "declared artifact not in the server's current listing" warning is a coordinated surface (whichever lands second adopts the other's vocabulary), not a dependency.
- Related: G-067 - the future .tm module artifact channel inherits this listing idiom (its artifact sets have the same growth shape); recorded here so the channel design adopts rather than reinvents.
- Related: G-006 - absorbs fetch/consent mechanisms when it lands; the listing mechanism must survive that absorption.
- Related: G-119 (archived) - the list -remote freshness/row vocabulary this goal extends.
- Related: G-105/G-130/G-116 - growth drivers (more targets, 32-bit lanes, tcltls artifacts) multiplying artifacts per tier; this goal keeps their listings bounded.
- Related: G-004 - the retention posture driver (immutable artifacts accumulate; nothing is deleted).
- Anchors (archived): G-117/G-103 (the -r<N> immutable-name pattern).
- Decision points settled in the work: the manifest filename (listing.txt the working name - it names more than "current": vulnerable/archive groups too); the exact reserved-group set (current/lts/vulnerable/archive the working vocabulary); whether tools/ normalizes to a top-level tier (tools/<platform>/) for symmetry with lib/<target>/ - the operator's recorded freedom to rearrange punkbin makes this cheap now.
- Posture (user direction 2026-07-31): no backwards-compat requirement for now (single corporate group, JN-controlled; compat becomes a concern with external consumers); no punkbin mirrors exist yet and the operator is free to rearrange the repo. A consent gate on vulnerable-group fetches (beyond the warning) is a possible later hardening, deliberately not in this goal.
- Overlap survey 2026-07-31 (drafting): goals_xref paths over src/scriptapps, src/tests/shell/testsuites/binscripts, src/make.tcl, bin - surfaced G-078/G-081 (src/scriptapps/tools doc tooling, domain-unrelated) and G-061 (binscripts testsuite path adjacency - pseudoconsole testing, domain-unrelated); no contract-level overlaps among live goals. Related set above from the domain sweep (punkbin layout + listing consumers + growth drivers).
Loading…
Cancel
Save