Browse Source
Per-tier listing manifests with named groups (current/lts/vulnerable/archive) separating the listing surface from the sha1sums integrity ledger; activation assessed as immediately possible (no goal dependencies; G-141 libfetch surface coordination only). Assisted-by: harness=opencode; primary-model=huggingface/moonshotai/Kimi-K3; api-location=unknownmaster
3 changed files with 53 additions and 0 deletions
@ -0,0 +1,45 @@
|
||||
# G-142 Punkbin curated listing manifests: per-tier artifact grouping with named groups |
||||
|
||||
Status: proposed |
||||
Scope: punkbin artifact repo (external c:/repo/jn/punkbin - layout contract, src/build_sha1sums.tcl, per-tier listing manifests); src/scriptapps/bin/ (punk-runtime payloads); bin/punk-runtime.cmd (regenerated output) + bin/AGENTS.md; src/tests/shell/testsuites/binscripts/; src/make.tcl (libfetch coordination) |
||||
Goal: punkbin artifact tiers (runtime platform folders, tools/, lib/<target>/) gain a curated per-tier listing manifest in the defaults.txt idiom (hand-curated, generator-validated, mirror-curable) that separates the listing surface from the integrity ledger: sha1sums.txt stays the complete integrity authority covering every retained immutable artifact, while a per-tier manifest names the listed subset in named groups with tool-understood semantics (current, lts, vulnerable carve-outs, operator-invented groups) - so an operator can retain old immutable artifacts indefinitely without punkshell-side listing tools drowning the current set in history, while every retained artifact stays fetchable and sha1-verifiable by explicit name. |
||||
Acceptance: (1) the punkbin layout contract (AGENTS.md + README) documents the per-tier listing manifest: file format (line-based, '#' comments/blanks ignored, two-column '<name> <group>'), the group vocabulary with reserved-group semantics (current = default listing; lts = listed in own section; vulnerable = hidden from default listing with a fetch-time warning; archive = hidden, fetch-silent; unknown groups listed as own sections; absent-from-manifest = retained-unlisted, fetchable by explicit name) and the mirror-curation posture; (2) build_sha1sums.tcl validates each manifest against its tier's sha1sums.txt (listed names must exist there) and reports per-tier listed/hidden/unlisted-retained counts; (3) punk-runtime list -remote renders the curated view by default (grouped sections; vulnerable + archive groups hidden) with an -all escape to the full sha1sums view, byte-agreeing across the ps1/bash payloads; (4) fetch of a vulnerable-group artifact succeeds with a warning naming the group; fetch of any retained unlisted artifact still verifies against the full sha1sums.txt; (5) binscripts characterization tests pin the listing/filtering/warning behaviours against the httpfixture server in both payloads; (6) libfetch warns when a declared artifact is absent from the server's current listing (surface coordinated with G-141). |
||||
|
||||
## Context |
||||
|
||||
The punkbin artifact repo's tiers are deliberately flat below their <arch> levels (runtime platform folders, tools/, lib/<target>/), and artifacts are immutable -r<N> names retained indefinitely by operator policy. Today sha1sums.txt is both the integrity authority AND the de-facto listing source: punk-runtime list -remote renders every row including remote-only entries, so retaining old revisions indefinitely grows the listing monotonically while the signal (what should I fetch?) drowns in history. defaults.txt already proves the curated-manifest idiom in this layout (hand-curated release decision, generator-validated, mirror-curable) but names only ONE artifact per platform - it cannot express "these N artifacts are the current set", nor named groups. |
||||
|
||||
User direction shaping the design (2026-07-31): grouping is a first-class need - anticipated groups include LTS tracks and carve-outs for releases with known vulnerabilities (still servable by explicit name, excluded from the main listing, warning on fetch); there are NO punkbin mirrors yet and the operator is currently free to rearrange the repo; backwards compatibility is NOT a requirement for now (single corporate group, JN-controlled - it becomes a concern only once punkshell/punkbin have external consumers). |
||||
|
||||
## Approach |
||||
|
||||
- Per-tier curated manifest (filename settled in the work - listing.txt the working name) beside each tier's sha1sums.txt, in the defaults.txt idiom: line-based, '#' comments/blanks ignored, two-column '<name> <group>'; hand-curated as a release decision in the publication change-set, never generated; mirrors may curate their own (defaults.txt precedent). |
||||
- Group vocabulary is OPEN with reserved-group semantics understood by tools: current = the default listing; lts = listed in its own section (a still-recommended track); vulnerable = hidden from the default listing, fetchable by explicit name with a fetch-time warning naming the group (the known-vulnerability carve-out - still servable, never silently listed); archive = hidden, fetch-silent (deliberately-retained history marker); operator-invented groups are listed as their own sections; names absent from the manifest are retained-unlisted - fetchable and sha1-verifiable by explicit name, never listed (the zero-curation bucket for bulk history). |
||||
- sha1sums.txt stays the COMPLETE integrity authority: every retained artifact, listed or not, stays fetchable by explicit name and sha1-verifiable. The manifest subsets listing, never integrity. |
||||
- build_sha1sums.tcl validates each manifest against its tier's sha1sums.txt (every listed name must exist there - the same drift discipline as defaults.txt) and reports per-tier listed/hidden/unlisted-retained counts as an audit surface. |
||||
- punk-runtime list -remote renders the curated view by default (grouped sections; vulnerable + archive groups hidden) with an -all escape to the full sha1sums view; fetch of a vulnerable-group artifact succeeds with a warning naming the group; fetch of unlisted artifacts verifies against the full sha1sums.txt unchanged. Both payloads agree byte-for-byte (the established ps1/bash parity discipline + re-wrap roundtrip test). |
||||
- libfetch warns when a declared artifact is absent from the server's current listing (drift signal for the declaring project; surface coordinated with G-141). |
||||
- Backwards compatibility is deliberately NOT a design constraint (user direction 2026-07-31). Tolerant defaults are kept only where free: an absent manifest means list-everything - the natural uncurated-server behaviour, which also serves future third-party mirrors using the layout. |
||||
|
||||
## Alternatives considered |
||||
|
||||
- Physical archive/ subfolder per tier - reconsidered given the recorded freedom to rearrange punkbin (no mirrors yet), still rejected: an artifact's URL should be as immutable as its bytes (locator stability for pinned references), and fetch-by-name stays a single fixed path per tier instead of a main-then-archive probe chain. Revisit only if actual pruning (not listing) becomes the need. |
||||
- Display-side grouping only (tools collapse old -r<N> revisions by parsed name series) - deferred complement, rejected as the mechanism: no operator curation (cannot express LTS tracks or vulnerability carve-outs) and sha1sums.txt keeps growing as the transfer unit. |
||||
- Sharded integrity (sha1sums.txt + sha1sums-archive.txt) - rejected: splits the integrity authority and forces every consumer to answer "which file do I verify against"; the manifest subsets listing, never integrity. |
||||
- Single root-level manifest covering all tiers (defaults.txt's own shape) - rejected: per-tier files sit beside the sha1sums.txt they subset (locality), let mirrors curate per tier, and keep each tier's curation surface proportional to its churn. |
||||
- Single-column names with '#' comment group headers - rejected: two-column '<name> <group>' is machine-renderable (grouped sections, reserved-group semantics) at the same curation cost; comment headers are human-only grouping. |
||||
|
||||
## Notes |
||||
|
||||
- Related: G-123 (achieved 2026-07-31 - see goals/archive/G-123-thirdparty-runtime-tiers.md) - runtime-tier twin; this goal's manifest sits beside its retroactive sidecars and schema-v2 records. |
||||
- Related: G-138/G-139/G-140 (archived) - the lib tier's emission/consumption; the manifest mechanism serves lib/<target>/ identically to runtime tiers, and libfetch is a consumer. |
||||
- Related: G-141 - libfetch target selection; the libfetch-side "declared artifact not in the server's current listing" warning is a coordinated surface (whichever lands second adopts the other's vocabulary), not a dependency. |
||||
- Related: G-067 - the future .tm module artifact channel inherits this listing idiom (its artifact sets have the same growth shape); recorded here so the channel design adopts rather than reinvents. |
||||
- Related: G-006 - absorbs fetch/consent mechanisms when it lands; the listing mechanism must survive that absorption. |
||||
- Related: G-119 (archived) - the list -remote freshness/row vocabulary this goal extends. |
||||
- Related: G-105/G-130/G-116 - growth drivers (more targets, 32-bit lanes, tcltls artifacts) multiplying artifacts per tier; this goal keeps their listings bounded. |
||||
- Related: G-004 - the retention posture driver (immutable artifacts accumulate; nothing is deleted). |
||||
- Anchors (archived): G-117/G-103 (the -r<N> immutable-name pattern). |
||||
- Decision points settled in the work: the manifest filename (listing.txt the working name - it names more than "current": vulnerable/archive groups too); the exact reserved-group set (current/lts/vulnerable/archive the working vocabulary); whether tools/ normalizes to a top-level tier (tools/<platform>/) for symmetry with lib/<target>/ - the operator's recorded freedom to rearrange punkbin makes this cheap now. |
||||
- Posture (user direction 2026-07-31): no backwards-compat requirement for now (single corporate group, JN-controlled; compat becomes a concern with external consumers); no punkbin mirrors exist yet and the operator is free to rearrange the repo. A consent gate on vulnerable-group fetches (beyond the warning) is a possible later hardening, deliberately not in this goal. |
||||
- Overlap survey 2026-07-31 (drafting): goals_xref paths over src/scriptapps, src/tests/shell/testsuites/binscripts, src/make.tcl, bin - surfaced G-078/G-081 (src/scriptapps/tools doc tooling, domain-unrelated) and G-061 (binscripts testsuite path adjacency - pseudoconsole testing, domain-unrelated); no contract-level overlaps among live goals. Related set above from the domain sweep (punkbin layout + listing consumers + growth drivers). |
||||
Loading…
Reference in new issue