indexbot Module Contracts (Phase 2 Build Wave)¶
This is the interface spec 16 parallel Sonnet builders implement against. It is
binding: implement these exact signatures. If a contract below is wrong,
underspecified, or blocks you, say so in your work package's open_questions —
never silently deviate. Frozen types (ports.py, model.py) already exist in
src/indexbot/; read them first, they are the ground truth for anything this
document summarizes rather than quotes verbatim.
Design authority, in order of precedence for anything this document doesn't
settle: adr_locked_observation_index_format.md (wire format, "ADR-1"),
adr_namespace_policy.md ("ADR-2"), adr_index_bot_and_workflow_security.md
("ADR-4"), adr_catalog_docs_colocation.md ("ADR-3"), plan_index_v1.md.
0. How "pure core/" actually works¶
core/ modules import nothing from adapters/ or httpx — but several take
a RegistryPort/ForgePort/FilePort/ClockPort argument directly (e.g.
core/observe.py's registry: RegistryPort parameter). This is not a
contradiction: "pure" here means deterministic given its explicit inputs,
including injected ports — a unit test passes a tests/fakes/ fake and gets
a 100%-deterministic result; production wiring (cli/main.py's eventual DI,
WP2-M) passes the real adapters/* implementation. "No I/O" means no direct
httpx/filesystem/time.time() call inside the module's own body — every
such effect is reached exclusively through an injected port. This is the same
pattern the existing scaffold already uses for ClockPort/FixedClock.
1. CAS objects are copied, not serialized (binding for every module below)¶
A CAS object under p/<ns>/<pkg>/o/sha256/<hex>.json is the OCI image index
the physical registry served for a tag, stored verbatim. Its ordering is
the registry's ordering; its whitespace is the registry's whitespace; <hex>
is sha256 of those exact bytes, which is the registry's own manifest digest
for that index. No module in this repo serializes one, so there is no
canonical encoding to agree on and nothing for two implementations to drift
apart about — the property earlier revisions bought with a sort key and a
minified encoder is now free, because the bytes never round-trip.
core/observe.py records ManifestFetch.raw as Observation.raw and
ManifestFetch.digest as Observation.content_digest; every writer
(cli/seed_import.py, core/render.py) copies that bytes object through
unchanged.
The one JSON document this bot does author is the package root, and it has
its own byte-exact form (§5.6, §14) — pretty-printed for PR review, never
digested. desc.digest comparisons and desc-blob digests are likewise
sha256 over bytes the registry served, never over a re-encoding.
2. Test conventions¶
- Golden fixtures (
core/render.py, WP2-F):tests/golden/render/<case>/, one directory per named scenario from ADR-4 BD-3's list (normal,orphan_pruned,yanked_excluded,shared_digest_dedup,no_desc,png_only_logo,nested_namespace). Each case directory holdsinput/(theSourcePackagefixtures, as plain Python data built in the test file — no need to serialize/deserialize JSON just to build a fixture) andexpected/dist/(the exact filesbuild_render_plan's returned tuple should produce, compared byte-for-byte). Keep fixtures as Python literals in the test module unless a case is large enough that a checked-in file genuinely reads better — most of these don't need one. - respx (
adapters/registry_v2.py,adapters/github_api.py, WP2-C/WP2-D): onerespx.mockroute per distinct response class per method (200/404/401-then-retry/429-with-Retry-After/5xx-exhausted/malformed-JSON). Assert on the port-level return/exception, not on respx call internals, so the test survives an adapter refactor. - hypothesis (
core/validate_entry.py'sparse_package_id, re-homed from the deletedcore/validate_payload.py):from_regex(PACKAGE_ID_RE, fullmatch=True)for the acceptance property; a second strategy seeded with.., absolute paths (/etc/passwd), and shell/format-string injection tokens for the rejection property; a wall-clock-bounded test (pytest-timeoutis not a declared dependency — usetime.monotonic()before/after a worst-case-length adversarial input and assert the elapsed time is under a fixed small bound, e.g. 50ms) proving the length cap makes regex work non-catastrophic even for a crafted 140-char input. - Idempotency (
core/regenerate.py/core/diff.py, WP2-B): "run twice, second diff empty" — callregeneratethendifftwice in a row with the same fakeObservationset and assert the seconddiffcall returnsNone, and that noTagEntry.observedtimestamp changed between the tworegenerateoutputs. - Everywhere: DAMP, self-contained per test — no shared fixture module beyond
tests/fakes/.
3. model.py / ports.py — already implemented, summary only¶
New since scaffold: OwnershipProbeResult, CommitStatusState,
PullRequestInfo (model.py); RegistryPort.get_blob/probe_ownership,
ForgePort.get_ref_sha/commit_files/get_pull_request_info/
set_commit_status, FilePort.read_bytes/write_bytes/list_files
(ports.py). Read the docstrings in those files — they are the exception
contract (which raises KeyError vs TransientError vs ValidationError)
and are not repeated here.
Further additions, fork-PR announce revamp (2026-07-18): PullRequestInfo
gains author_login: str = ""/author_id: int = 0 (defaulted so every
pre-existing classify_pr-only construction site stays unchanged — only
cli/governance_check.py's G-19 gate needs both set for real).
ForgePort.open_or_update_pull_request gains an optional
head_owner: str | None = None keyword (cross-repo/fork-PR head,
f"{head_owner}:{branch}", vs. the same-repo plain branch).
ForgePort gains request_reviewers(pr_number, logins),
create_comment(pr_number, body, *, marker) (idempotent via a hidden HTML
marker), and create_or_update_issue(*, title, body, labels=None) (promoted
from an adapter-only capability — see §13 item 4). All three implemented in
adapters/github_api.py::GitHubApi and tests/fakes/__init__.py::FakeGitHub.
Types referenced below that are not in model.py (cross-core/-module
data, deliberately kept out of the port-boundary file — see ports.py's
module docstring) must be defined by the owning module as a
@dataclass(frozen=True, slots=True) exactly as shaped here.
4. core/validate_payload.py — removed (fork-PR announce revamp, 2026-07-18)¶
This module and its dedicated test file no longer exist. PACKAGE_ID_MAX_LENGTH,
_NAMESPACE_MAX_LENGTH, _PACKAGE_MAX_LENGTH, _NAMESPACE_SHAPE,
_PACKAGE_SHAPE, PACKAGE_ID_RE, and parse_package_id re-homed verbatim
(zero behavior change) into core/validate_entry.py — see §5, which is now
the single home for both the two-segment package-id grammar and the
N-segment OCI repository grammar (BD-4's two-regex rule still holds: the two
constants stay structurally distinct, only their file is shared now). The
original rationale for a standalone module — parse_package_id was reached
via cli/_common.py's read_validated_env, the repository_dispatch
PACKAGE_ID env-var-indirection reader (ADR-4 BD-4) — no longer applies:
the doorbell pipeline (env var, --validate-only) retired entirely in the
revamp (owner-confirmed decision set 2026-07-18,
"Fork-PR announce": publishers open PRs from forks under their own GitHub
identity, no index-side credentials, no doorbell). read_validated_env
itself is deleted from cli/_common.py along with its tests — every
remaining caller of parse_package_id (cli/validate.py,
cli/reconcile.py, cli/seed_import.py) now imports it from
core/validate_entry.py directly.
5. core/validate_entry.py (WP2-E)¶
_COMPONENT = r"[a-z0-9]+(?:(?:\.|_|__|-+)[a-z0-9]+)*"
OCI_REPOSITORY_RE: Final[re.Pattern[str]] = re.compile(rf"^{_COMPONENT}(?:/{_COMPONENT})*$")
# G-03's allowlist is NOT a constant here — it is this deployment's committed
# policy (§15, `.github/index-policy.json`), loaded by `cli/_wiring.py` and
# passed in. Still "extend only via reviewed PR": the policy is a committed
# file, never an environment or Actions variable.
OCI_REPOSITORY_RE is a structurally distinct constant from
PACKAGE_ID_RE — never share a compiled pattern or a "guess which shape"
helper between the two (ADR-4 BD-4, the regclient/regsync failure mode).
Functions (each raises ValidationError on failure, never returns a bool):
check_name_matches_path(package_id: PackageId, root: PackageRoot) -> None— G-02:root.name == f"ocx.sh/{package_id.namespace}/{package_id.package}".check_superseded_by(root: PackageRoot) -> None— no-op whenroot.superseded_by is None; otherwise the value must shape-validate as a<namespace>/<package>id (reusing this module's ownparse_package_id— never a second hand-rolled regex) and must not namerootitself. Deliberately does not checkroot.statuscoupling (a package can name a successor while stillactive) nor whether the named successor exists or is reserved (a dangling/not-yet-claimed successor is allowed, likedeprecated_message's free-text pointer).check_repository_allowlisted(repository: str, allowed_hosts: frozenset[str]) -> None— G-03. Parses theoci://<host>/<path>URI (stdliburllib.parse, no regex needed for the scheme/host split) and checkshost in allowed_hosts.allowed_hostsis this deployment's committed registry-host policy (§15), a required argument with no default — no caller can run G-03 against a policy nobody stated, and the public index'sghcr.iois not a corporate copy's Harbor host. Must run before anyRegistryPortcall — SSRF ordering, BD-1.check_repository_shape(repository: str) -> None— validates the<path>portion ofoci://<host>/<path>againstOCI_REPOSITORY_RE(N-segment grammar — neverPACKAGE_ID_RE).parse_digest(raw: str) -> str—re.fullmatch(r"sha256:[a-f0-9]{64}", raw)orValidationError. Every digest-shaped string anywhere in the bot (TagEntry.content, an image index'smanifests[*].digest,Desc.digest/.readme/.logo) is validated through this one function before it is ever used to build a filesystem path — digest-hexfullmatchbefore path join, no exceptions.check_digest_self_consistent(digest: str, object_bytes: bytes) -> None(fork-PR announce revamp, 2026-07-18) — the general form: recomputes sha256 ofobject_bytes(the committed bytes exactly as they sit on disk — a byte-equality check, never a re-serialization) and compares todigest; mismatch isAnomalyError(this is CAS integrity, not a routine validation failure — the file's name lies about its own content). Any claimed digest string works here, not only aTagEntry's —cli/validate.py's blanket per-file CAS scan andcore/verify_claims.py's desc-blob hash check both need this, closing the byte-exact-discipline gap where only tag digests were ever verified.check_content_digest_self_consistent(tag: TagEntry, object_bytes: bytes) -> None— thinTagEntry-shaped wrapper overcheck_digest_self_consistent(tag.content, object_bytes), kept for its pre-existing callers/tests.check_no_dangling_references(root: PackageRoot, cas_digests: frozenset[str]) -> None— everyTagEntry.contentandDesc.readme/Desc.logo(whendescis notNone) must appear incas_digests(the set of digests actually present under this package'so/sha256/tree, as enumerated by the caller viaFilePort.list_files). RaisesAnomalyErrorper missing reference — a root pointing at a CAS object that doesn't exist is corruption, not a routine PR mistake.parse_package_root(raw: bytes) -> PackageRoot/serialize_package_root(root: PackageRoot) -> bytes— thedict<-> dataclass codec every other module reuses (§1).serialize_package_rootproduces the exact bytes committed top/<ns>/<pkg>.json— pretty-printed (json.dumps(..., indent=2, sort_keys=False)preserving the field ordermodel.PackageRootdeclares them in, matchingschema/root.schema.json'srequiredorder) plus a trailing newline. It is the one JSON document this bot authors, and it is optimized for PR review, not digest stability — the root's own bytes are never digested, only referenced byTagEntry.content, which points at an OCI image index, not at the root itself.upstream: None-> the"upstream"key is omitted from the dict entirely (schema forbidsnullthere, ADR-2 ND-9);superseded_by: None-> the"superseded_by"key is likewise omitted entirely (same omit-when-absent contract);desc: None->"desc": nullis written (schema requires the key, allowsnull, ADR-1 D6).parse_package_rootraisesValidationErroron any structurally malformed input (missing required key, wrong JSON type) — it does not re-validate shape-schema concerns already covered bycheck-jsonschema(regex patterns, enum membership); it only needs to not crash on well-formed-but-unexpected JSON and to fail loudly (never partially construct aPackageRoot) on malformed JSON.is_reserved_tag(tag: str) -> bool/check_no_reserved_tags(root: PackageRoot) -> None— D7's tag reservation, one implementation, two callers. Reserved: the case-insensitive__ocxprefix (__ocx.desc,__ocx,__ocxfoo,__OCX.desc), and the canonicalsha256.<64hex>/sha384.<96hex>/sha512.<128hex>tagsocx package pushwrites (hex case-insensitive, per-algorithm length exact).check_no_reserved_tagsraisesValidationErrorlisting every offendingtagskey — the PR gate is the only layer a hand-authored root passes through.core/observe.py's sweep importsis_reserved_tagto exclude such tags rather than refuse the repository;schema/root.schema.json'spropertyNames.notdocuments the same intent but cannot express the full rule.parse_image_index_digests(raw: bytes) -> tuple[str, ...]— the D4(c) document-kind gate. One committed CAS object'smanifests[*].digest, in wire order;ValidationErrorifrawis not a JSON object carrying amanifestslist of descriptors with stringdigestfields. There is no write side: nothing serializes a CAS object (§1). Unknown index fields (subject,artifactType,annotations, future spec additions) are passed over, not rejected — these are bytes OCX does not author.
registry_checks (network — G-15, digest-scope):
check_digest_in_scope(repository: str, digest: str, registry: RegistryPort) -> None—registry.get_manifest(repository, digest); aKeyError(404) means the claimed content digest does not actually exist on the physical repo -> re-raise asValidationError(a claim about registry content that isn't true is a validation failure, not an anomaly — nothing was ever legitimately observed to mutate).check_ownership(repository: str, expected_name: str, registry: RegistryPort) -> OwnershipProbeResult— thin pass-through toregistry.probe_ownership. The caller (cli/validate.py) decides disposition:"mismatch"->ValidationError(block);"unconfirmed"-> do not raise — return the result so the caller can attach a WARN annotation to the PR (ForgePort.add_labelswith something likeownership-unconfirmed, or a PR comment —cli/validate.py's call, WP2-H..L). Never silently treat"unconfirmed"as"confirmed".
6. core/version_order.py (WP2-F)¶
Ported from ocx/scripts/catalog-generate.py's find_latest_version (real
source read for this stage — verified no separate "yank-exclusion" code
exists there; that logic is new, per ADR-1's yank semantics, not a port):
_VERSION_RE: Final[re.Pattern[str]] = re.compile(
r"^(?:([a-z][a-z0-9.]*)-)?((0|[1-9][0-9]*)(?:\.(0|[1-9][0-9]*)(?:\.(0|[1-9][0-9]*))?)?)$"
)
def is_build_pinned_version(tag: str) -> bool:
"""True iff `tag` parses as an OCX version (`_OCX_VERSION_RE`, the whole
grammar `Version::parse` spells, `latest` refused as a variant prefix)
AND carries a build fragment: `3.28.1_20260216`, `slim-3.12.13_20260728`,
`1.0.0-rc1_20260728`. The build fragment is what makes a tag immutable —
`ocx package push` writes it once and repoints every rolling ancestor at
it. `latest`, a bare major (`3`), `3.28`, `3.28.1`, a bare variant name,
and any opaque tag are all `False`: those are the cascade targets, and
moving them is what a publish *is*. See `core/anomaly.py` (§7).
"""
def find_latest_version(tags: Mapping[str, TagEntry]) -> str | None:
"""Highest version among tags that are (a) not "latest", (b) unprefixed
(`m.group(1) is None` — variant tags are skipped, matching the ported
function's original behavior verbatim), and (c) not yanked
(`tags[t].yanked is None` — new: ADR-1 yank semantics, a yanked tag must
never be selected as the displayed/default version). Comparison is by
the parsed `(major, minor, patch)` int tuple, missing components treated
as absent (not zero) for tuple comparison purposes, matching the ported
function's `tuple(int(x) for x in m.group(2).split(".") if x)` behavior
exactly. Returns `None` if no eligible tag exists.
"""
7. core/observe.py / core/regenerate.py / core/diff.py / core/anomaly.py / core/desc.py / core/backoff.py (WP2-B)¶
These six ship together (one work package) but are listed separately since several are consumed by other WPs built in parallel.
core/backoff.py¶
@dataclass(frozen=True, slots=True)
class BackoffPolicy:
max_attempts: int = 5
base_delay_seconds: float = 1.0
max_delay_seconds: float = 30.0
def is_retryable_status(status_code: int) -> bool:
"""True for 429 or any 5xx. False for everything else, including other
4xx (401/404 are permanent failures for a given request, never retried
by this policy — 401 gets one token-refresh-and-retry inside
`adapters/registry_v2.py`, which is a different mechanism, not backoff)."""
def delay_seconds(
attempt: int, policy: BackoffPolicy, *, jitter: float, retry_after: float | None = None
) -> float:
"""`attempt` is 1-indexed. If `retry_after` is given and positive, it
wins outright (the server said exactly how long to wait — G-10).
Otherwise: `min(policy.max_delay_seconds, policy.base_delay_seconds * 2 ** (attempt - 1)) * (0.5 + jitter)`,
`jitter` in `[0, 1)` supplied by the caller (`adapters/registry_v2.py` passes
`random.random()`; tests pass a fixed float) — keeps this function
itself deterministic and trivially 100%-coverable without mocking
`random` or `time`.
"""
The retry loop (attempt counting, calling httpx, sleeping, deciding
when policy.max_attempts is exhausted and raising TransientError) lives
in adapters/registry_v2.py — it is imperative-shell code that happens to consult
this pure module's two functions for its decisions. Do not move the loop
into core/backoff.py; that would require mocking time.sleep/httpx to
test it, defeating the whole point of the split.
core/observe.py¶
@dataclass(frozen=True, slots=True)
class Observation:
"""A record of what a tag resolved to at observation time. `raw` is the
registry's OCI image index, verbatim — this class names the *event*,
never the artifact. Input to regenerate/anomaly."""
tag: str
content_digest: str # == ManifestFetch.digest, the registry's own index digest
raw: bytes # == ManifestFetch.raw, never re-serialized
source: str | None = None # org.opencontainers.image.source, https:// only
def observe_one_tag(repository: str, tag: str, registry: RegistryPort) -> Observation | None:
"""One tag's freshly observed state, or `None` if `tag` no longer exists
on `repository` (a real 404). The fetched manifest must be an OCI image
index — discriminated by a `"manifests"` key — or `ValidationError` is
raised naming both the tag and the repository: this index records image
indices only, so a tag resolving to a single image manifest is a
publishing fault to surface, not a shape to convert. Extracted (fork-PR
announce revamp, 2026-07-18) so a caller that already knows which
*specific* tags it cares about — `core/verify_claims.py` re-deriving one
claimed tag, a publisher's own tool observing only its curated tag set —
never has to call `registry.list_tags()` first just to reach a single
tag's manifest.
"""
def observe(repository: str, registry: RegistryPort) -> tuple[Observation, ...]:
"""One `Observation` per `registry.list_tags(repository)` entry, via
`observe_one_tag`, **skipping every reserved tag name**
(`validate_entry.is_reserved_tag`, §5.6 — imported, never restated).
That exclusion is load-bearing: `ocx package push` writes a canonical
`sha256.<hex>` tag beside every version tag plus an `__ocx.desc`
description tag, both resolving to bare image manifests, so a sweep that
did not skip them would refuse every ocx-published repository on its
first reserved tag. A tag whose manifest fetch raises `KeyError` (fetched
but vanished between `list_tags` and `get_manifest` — a real registry
race) is **skipped**, not fatal — `observe_one_tag` returning `None`. A
`TransientError` from either call propagates uncaught (the whole
`observe()` call fails transient, per BD-2 — no partial-tag
silently-skipped-on-backoff-exhaustion semantics; that's different from
the vanished-tag case above, which is a real 404, not exhausted
backoff).
"""
core/verify_claims.py (fork-PR announce revamp, 2026-07-18 — new)¶
FindingKind = Literal[
"tag-missing-upstream", "digest-mismatch",
"cas-object-missing", "cas-object-hash-mismatch",
"desc-blob-missing", "desc-blob-hash-mismatch",
]
@dataclass(frozen=True, slots=True)
class ClaimFinding:
package_id: PackageId
kind: FindingKind
detail: str # claimed tag name, or "desc.readme"/"desc.logo"
def verify_claims(
package_id: PackageId,
root: PackageRoot,
cas_object_bytes: Mapping[str, bytes],
registry: RegistryPort,
*,
base: PackageRoot | None = None,
) -> tuple[ClaimFinding, ...]:
Re-derives every claimed tag/desc-blob digest in root individually from
registry truth — subset semantics, never a full-set equality against
registry.list_tags() (an owner's curated tags map may legitimately be a
subset of what the registry carries; that is the entire point of owner
curation — decision-set item 2, "announce is the only add/remove
authority"). Per tag: observe_one_tag(root.repository, tag, registry)
returning None -> "tag-missing-upstream"; a different content_digest
-> "digest-mismatch"; observe_one_tag raising ValidationError (the
tag now resolves to a bare image manifest, or to bytes over
_MAX_INDEX_BYTES) -> "digest-mismatch" as well, since the claim that
this tag resolves to the committed index is exactly what stopped being true;
the claimed digest missing from/not hashing to
cas_object_bytes -> "cas-object-missing"/"cas-object-hash-mismatch".
root.desc.readme/.logo, when set, get the identical CAS-hash check
(missing/mismatch -> "desc-blob-missing"/"desc-blob-hash-mismatch") —
closing the gap where only tag digests were ever byte-verified (byte-exact
discipline). Pure — returns findings, never raises; the caller decides
disposition. Two callers, two dispositions for the same taxonomy:
cli/validate.py's unprivileged PR gate treats any finding as a
ValidationError (reject the PR — nothing was ever legitimately observed to
mutate, the claim just isn't true right now); cli/reconcile.py's
verify-only nightly sweep escalates every finding kind to its AnomalyError
exit-65 outcome except a "digest-mismatch" on a floating (non-pinned)
tag and a "tag-missing-upstream" on a yanked tag (ADR-6 FP-2/FP-3 — yank
is grace, an explicit owner-authorized exemption from the
registry-existence check; everything else vanished-upstream is an anomaly,
not a silent drop) — see §12's cli/reconcile.py entry for the full
disposition table: floating-tag drift is expected cascade behavior, and a
still-pinned-tag mutation is caught by the reused
core/anomaly.py::check_tag_mutations instead, not by verify_claims.
base (0.5.1) is the same root as it stands on the pull request's base
ref, and it scopes the two REGISTRY checks — "tag-missing-upstream" and
"digest-mismatch" — to the claims the pull request actually makes. A
tag -> content pair byte-identical to the base ref is not asserted by this
pull request: it passed this same gate when it landed, and whether it is
still true today is cli/reconcile.py's question, which the paragraph
above answers differently on purpose. Without it, a pull request touching
only governance metadata is rejected for upstream drift in tags it never
mentioned — ocx-sh/index's forge-neutral owners[] rename failed on
twelve of 124 packages whose latest/partial-version tags had moved since
their last announce.
The narrowing is registry-only and cannot hide a hostile edit: the local CAS
byte checks ("cas-object-missing"/"cas-object-hash-mismatch") run on
every claim, carried over or not; a tag the pull request adds or repoints
has no byte-identical base entry and is verified in full; a repository
differing from the base ref discards the whole carried-over set (every tag
then resolves against a different physical registry, so no claim under it
was ever verified); and base=None — no --base-dir, no such root at the
base ref, or base bytes this version cannot parse — verifies everything.
cli/validate.py::_base_root is that read, done once per validated root and
shared with ND-4's update-vs-claim predicate.
core/desc.py¶
Ground truth for the __ocx.desc artifact (read from ocx/crates/ocx_lib's
oci/client.rs::pull_description and oci/annotations.rs — not guessed):
- Tag name: literal
"__ocx.desc". - Manifest: a single OCI image manifest (never an image index) with
artifactType == "application/vnd.sh.ocx.description.v1". manifest.layers[]: exactly one layer withmediaType == "application/markdown"(the readme — required, a description manifest with no markdown layer is malformed) and at most one layer withmediaType"image/png"or"image/svg+xml"(the logo — optional). Layer content is fetched viaRegistryPort.get_blob(repository, layer.digest).manifest.annotations(manifest-level, not layer-level):org.opencontainers.image.title(title),org.opencontainers.image.description(description),sh.ocx.keywords(comma-separated string — split on,, strip whitespace, drop empty segments, matchingocx/scripts/catalog-generate.py'sparse_keywordsexactly).- Readme/logo bytes are copied verbatim — no frontmatter re-parsing (that
machinery,
ocx_lib::package::description::parse_readme, is publish-side only; the index bot only ever fetches).
@dataclass(frozen=True, slots=True)
class DescUpdate:
"""Non-`None` return of `check_desc_change` — what the caller persists."""
desc: Desc
readme_bytes: bytes | None
logo_bytes: bytes | None
def check_desc_change(
repository: str, current: Desc | None, registry: RegistryPort, *, name: str
) -> DescUpdate | None:
"""Compares `registry.get_desc_tag_digest(repository)` against
`current.digest` (or `None` if `current is None`). Returns `None`
(no change — caller keeps `current` verbatim, writes nothing new) if
they match, including both-absent. Otherwise fetches the `__ocx.desc`
manifest and its layers per the format above, builds the new `Desc`
(`digest` = the observed `__ocx.desc` tag digest itself, not a
recomputed content hash — this is a floating-tag comparison, D6, not a
CAS digest), and returns a `DescUpdate` whose `readme_bytes`/
`logo_bytes` the caller writes as this package's new CAS objects at
`o/sha256/<hex>.<ext>` (`hex` = sha256 of those exact bytes per §1;
`.md` for the readme, `.svg`/`.png` for the logo per its layer media
type). `desc.readme`/`desc.logo` in the returned `Desc` are those same
`sha256:<hex>` digest strings. A missing logo layer -> `logo_bytes = None`,
`desc.logo = None`. A missing `sh.ocx.keywords` annotation ->
`desc.keywords = ()`.
`name` is the entry's logical name, used only for the title fallback.
"""
desc.titleis never the empty string. Theorg.opencontainers.image.titleannotation is optional on the publisher's side, butschema/root.schema.jsongivesdesc.titleminLength: 1, and nothing validates a realp/**root against that schema untilschema:validate:renderedruns after merge — where a violation blocks the site deploy for every package. The fallback chain is annotation, then the last/-segment ofname, then ofrepository. It is the same chainocx'sannounce::pipeline::titleapplies, and the parity is load-bearing: two tools that disagree on a title write different roots for identical registry state, so each would see the other's root as changed and the C6 unchanged-is-a-no-op short-circuit would never settle.
core/regenerate.py¶
def regenerate(
current: PackageRoot, observations: tuple[Observation, ...], desc: Desc | None, clock: ClockPort
) -> PackageRoot:
currentis required, neverNone— a package_id with no committed root is a validation error the caller (cli/reconcile.py,cli/seed_import.py) raises before callingregenerate(namespace claiming, ADR-2 ND-5, is a separate human-PR flow that already commits a root with emptytagsbefore the firstannounceever runs —regeneratenever synthesizes a root from scratch).- Human-governed fields (
name,repository,owners,status,deprecated_message,created,upstream,superseded_by) are carried over verbatim fromcurrent— never regenerated (G-09). desc: passcurrent.descunchanged whencore/desc.pyfound no change, or the newDescfrom a non-NoneDescUpdate.descwhen it did.regeneratedoes not callcore/desc.pyitself — the caller composes both.tags: rebuilt entry-by-entry fromobservations. A tag whosecontent_digestequalscurrent.tags[tag].contentkeeps that entry'sobservedtimestamp unchanged (no gratuitous timestamp churn on a no-op re-observe — this is what makes "run twice, second diff empty" hold, §2's required idempotency test). A new or changed-content tag getsobserved = clock.now_iso8601(). A tag present incurrent.tagsbut absent fromobservations(removed upstream) is dropped.yanked: an existingTagEntry.yankedmarker survives untouched (human-governed, G-05) even if that tag's content also changed this run.
Open question (neither ADR states this explicitly): does a
re-published digest under a yanked tag name clear the yank? This
contract's default is no — preserve yanked regardless of content
change. Confirm with the owner before Phase 3.
- A tag vanishing from the registry entirely (present in current, absent
from observations) is not itself an anomaly — core/anomaly.py
only checks digest mutation on a still-present pinned tag, not
disappearance. Open question: is silent tag disappearance actually
fine, or should reconcile flag it too? Not decided by either ADR; flagged
here rather than silently assumed safe.
core/diff.py¶
@dataclass(frozen=True, slots=True)
class Patch:
package_id: PackageId
root: PackageRoot # target — write verbatim (validate_entry.serialize_package_root)
new_objects: tuple[
tuple[str, bytes], ...
] # (digest, the registry's index bytes) not already reachable from `current`
summary: str # one-line PR-body fragment, e.g. "+3.29.0, ~latest -> sha256:bbbb"
def diff(current: PackageRoot, target: PackageRoot) -> Patch | None:
"""`None` iff `current == target` structurally (dataclass equality —
both are frozen, so this is a plain `==`) — BD-2's `ExitCode.OK` no-op
case. Otherwise a `Patch`. `new_objects` is target's tags whose content
digest does not appear anywhere in `current.tags` — already-existing
objects (shared digest / cascade aliasing, ADR-1 D3) are excluded so a
publisher never re-writes a CAS object that's already committed.
"""
ChangeClass = Literal["new-package", "refresh", "human-review-required"]
def classify_change(before: PackageRoot | None, after: PackageRoot) -> ChangeClass:
"""`cli/classify_pr.py`'s core. `before` is the base-ref root, `None` if
the PR added a brand-new `p/<ns>/<pkg>.json` (the path did not exist at
the base ref — G-04). `before is None` -> always `"new-package"`.
Otherwise the machine lane is narrow by design (fork-PR announce revamp,
2026-07-18): **any field outside `tags`/`desc` changing is
`"human-review-required"`** — every governance field
`product-context.md` lists is human-authored, never auto-mergeable.
Concretely: `repository`, `owners`, `status`, `deprecated_message`,
`created`, `upstream`, or `superseded_by` differing -> `"human-review-required"`,
OR any tag present in both `before.tags` and `after.tags` has a
different `yanked` value (G-05's expanded key set, ADR-4 disposition
table) — else `"refresh"`. `name` is not checked here (pinned by
`check_name_matches_path` instead — a structural invariant, not a
governance-vs-machine distinction); `desc` is not checked here either
(bot-derived from the registry's `__ocx.desc` tag, `core/desc.py` — not
human-authored, stays in the machine lane alongside `tags`).
"""
core/anomaly.py¶
@dataclass(frozen=True, slots=True)
class AnomalyFinding:
package_id: PackageId
tag: str
committed_content: str
fresh_content: str
def check_tag_mutations(
package_id: PackageId, committed: PackageRoot, fresh: tuple[Observation, ...]
) -> tuple[AnomalyFinding, ...]:
"""Empty tuple = clean. For every tag present in both `committed.tags`
and `fresh` that `core/version_order.is_build_pinned_version` classifies
`True` (pinned — a version carrying a build fragment,
`3.28.1_20260216`), a different content digest between `committed` and
`fresh` is one `AnomalyFinding`. Tags classified `False` — `latest`,
`3`, `3.28`, `3.28.1`, a bare variant name, any opaque tag — are the
rolling cascade targets and are never flagged regardless of digest
change: moving them is what a publish *is* (ADR-1 D2/D3,
`crates/ocx_lib/src/package/cascade.rs`).
**Resolved 2026-07-29** (was §13 item 3). The predicate shipped as the
exact inverse of this — it checked `X.Y.Z` and skipped `X.Y.Z_<build>`,
which `_VERSION_RE` could not even express. On the live index that left
all 49 immutable tags exempt and all 71 checked tags ones the cascade is
supposed to move: a force-repointed build tag swept clean, and the next
legitimate republish of any package would have filed a tamper issue
against its own rolling tags.
Rolling tags stay exempt outright rather than getting a weaker check
(forward-only, or "must land on a digest some build tag also carries").
Neither is decidable from what the sweep observes: it re-observes only
the tags the committed root already claims, so the newly published build
tag a legitimate cascade points at is not in the observation set, and
tag ordering is not carried either. Doing it properly means listing tags
from the registry — a different sweep, not a tightening of this one.
"""
Returning findings (not raising) lets cli/reconcile.py implement the
plan's "partial-success semantics" (clean-subset PR + one anomaly issue
listing every finding + exit 65) — check_tag_mutations itself never
raises AnomalyError; the CLI layer maps a non-empty result to that outcome.
8. core/render.py (WP2-F, reshaped by plan_site_redesign WP-bot, then¶
by the @ocx-sh/catalog extraction, plan_catalog_extraction WP-11)
core/catalog_md.py — this section's original wrapper-page-Markdown
module — is deleted. plan_site_redesign retired bot-generated
per-package wrapper pages in favor of dynamic routes
(site/src/[ns]/[pkg].paths.ts, globbing the committed p/*/*.json tree
directly at VitePress build time — see adr_catalog_docs_colocation.md
Amendment A1). plan_catalog_extraction WP-11 then retired those dynamic
routes too — per-package pages are now synthesized by @ocx-sh/catalog's
own build engine (cat/src/build/pages.ts) from the wire tree, not by
anything under site/. core/render.py now emits exactly one output tree;
cas_relpath (the CAS path-building helper the deleted module also used)
relocated to core/validate_entry.py, alongside cli/reconcile.py's
existing import of it.
WP-11 update: the catalog-grid view-model (/data/catalog/catalog.json,
previously emitted here — see the retired shape documented below for
provenance) is retired from this module. That projection now lives
entirely in the @ocx-sh/catalog npm package's own view-model emitter
(cat/src/viewmodel/), a byte-gated TS port of this module's former
_catalog_platforms/_latest_activity/_catalog_entry/
_generated_timestamp/_catalog_index functions, which reads the wire
tree this module still produces (config.json, /p/**, c/index.json)
and renders the catalog UI directly from it — no bot-emitted view-model
JSON in between any more. core/render.py now emits ONLY the wire tree.
@dataclass(frozen=True, slots=True)
class SourcePackage:
"""One package's fully-loaded source-tree state — cli/render.py's input
unit, assembled via FilePort reads (list_files over `p/`, read_text per
root, read_bytes per CAS object)."""
package_id: PackageId
root: PackageRoot # parsed — drives the reachability walk
root_raw: bytes # exact p/<ns>/<pkg>.json source bytes — copied verbatim into dist, never re-serialized
content_by_digest: dict[str, bytes] # digest -> raw CAS bytes, this package's CAS only (key/extension bookkeeping is WP2-F's internal choice — see note below)
@dataclass(frozen=True, slots=True)
class FileWrite:
path: str # relative to the dist output root (`--out`)
content: str | bytes
def build_render_plan(packages: Sequence[SourcePackage], *, format_version: int = 1) -> tuple[FileWrite, ...]:
Pure (§0). No RenderPlan wrapper dataclass any more — build_render_plan
returns the flat dist-tree file list directly, since there is only ever one
output tree (--out, applied by cli/render.py after site:build's
VitePress build completes — see taskfile.yml's render:build task and its
emptyOutDir footgun comment).
Reachability walk per package: only tags[*].content digests of live
(non-yanked) tags, and, transitively, desc.readme/desc.logo digests, are
copied — CAS objects orphaned by a repointed or yanked tag are pruning
candidates (ADR-1 D8, deployment artifact only, never source-tree git
history). A yanked tag's content is pruned only if unreachable from every
other live tag — yanking does not itself force pruning while another tag
still shares the digest (emergent aliasing, ADR-1 D3, applies to
reachability too).
Returned file list:
- config.json: {"format_version": format_version, "name_segments":
NAME_SEGMENTS}. D7's "nothing else, ever" governs what a client must be
able to act on — the version pin is the only gate — not the literal key
count; name_segments publishes the name shape this deployment can hold so
a client need not probe for it. Both keys are emitted unconditionally.
- One p/<namespace>/<package>.json per package: content = source.root_raw
verbatim (never re-serialize through the dataclass — see §5's rationale).
- Every reachable p/<namespace>/<package>/o/sha256/<hex>.<ext> — copied
verbatim from content_by_digest.
- c/index.json: {"format_version": format_version, "packages": {"<ns>/<pkg>": "sha256:<hex>", ...}}
— the versioned envelope, never a bare {"<ns>/<pkg>": ...} map at the
document root. The listing lives under packages; format_version is the
same pin config.json carries, so a catalog names its own grammar. (A
client that read the root object as the listing itself is reading a shape
this bot has never emitted — ocx-sh/ocx did exactly that until 2026-07-27,
which is why this sentence exists.) One entry per package in ordered,
keyed on the bare <namespace>/<package> id (not the ocx.sh/-prefixed
name). The digest is sha256 of
source.root_raw's exact committed bytes — explicitly not a
re-serialization through serialize_package_root (which would be
byte-identical today and is still the wrong input: what this digest
attests is the file that was committed, not what the dataclass would
produce from it).
There is no fourth output any more — no data/catalog/catalog.json. That
shape (frozen by plan_site_redesign, referencing logo/readme blobs by CAS
URL rather than duplicating blob bytes, generated = lexicographic max over
every tag's observed/yanked.at, packages[] sorted by package id) is
retired from this module per the WP-11 update above; its authoritative
definition now lives in the @ocx-sh/catalog package's own viewmodel
contract (cat/'s own design docs), not here.
Note on content_by_digest keying: a CAS digest alone does not carry its
file extension (.json vs .md vs .svg/.png) — the extension is only
known from the filename cli/render.py discovers via FilePort.list_files.
Key the map however is convenient (e.g. "sha256:<hex>.<ext>", or a
(digest, ext) tuple) as long as the reachability walk itself keys purely
on the bare sha256:<hex> digest strings stored in TagEntry.content /
Desc.readme / Desc.logo — that part is frozen, the key encoding is not.
9. adapters/registry_v2.py (WP2-C)¶
Implements RegistryPort for any OCI Distribution (Registry v2) host. One
RegistryV2 client per host — ghcr.io and ocx.sh differ only in the URL
that issues pull tokens (https://ghcr.io/token vs. the Artifactory realm
OCX_SH_REALM; the default is <base_url>/token) — with
RoutedRegistry picking a client per call from the oci://<host>/… URI, so
core/ still sees exactly one RegistryPort. cli/_wiring._registry()
builds the mapping and _wiring.REGISTRY_ADAPTER_HOSTS names its keys.
Bearer-token dance: anonymous pull tokens
via GET <realm>?service=<host>&scope=… — fetch once per repository, cache for the
adapter instance's lifetime, refresh once (not counted against
BackoffPolicy.max_attempts) on a single 401, fail with TransientError on
a second consecutive 401 for the same request (a persistent auth failure is
not a backoff-retryable condition, but is also not a ValidationError — the
adapter couldn't complete the read, full stop).
Manifest/blob fetch retry loop (the imperative-shell half of §7's
core/backoff.py split): on each httpx call, if the response status
satisfies backoff.is_retryable_status, sleep
backoff.delay_seconds(attempt, policy, jitter=random.random(), retry_after=parsed_retry_after_header)
(via time.sleep) and retry, up to policy.max_attempts; on exhaustion
raise TransientError.
The same loop and the same attempt budget cover httpx.TransportError —
read/connect timeout, connection reset, protocol error — raised by the call
itself, including the nested token fetch. These are exceptions, so
is_retryable_status never sees them; without an explicit arm they escape
the adapter and reach cli/main.py, which only maps IndexBotError onto an
exit code and a step summary, so a network blip fails a run with a bare
traceback and no ExitCode.TRANSIENT (observed 2026-08-03 on a REQUIRED
schema-validate-pr check). They carry no server-supplied Retry-After, so
the delay is always the exponential/jitter form. A transport failure during
the token fetch leaves the 401 lane re-armed — the retry would otherwise
send unauthenticated, 401 again, and trip "persistent 401" instead of
spending its budget.
A malformed-JSON body on an otherwise-200 response
is not retryable — raise a plain ValueError-derived parse error
(propagates as an unhandled bug per cli/main.py's contract, since a 200
with unparseable JSON from GHCR is not a condition the bot has a defined
recovery for).
list_tags: paginate GHCR's tags/list?n=&last= — bounded pagination (a
hard cap, e.g. 10,000 pages, converted to TransientError if ever hit,
rather than an unbounded loop).
10. adapters/github_api.py (WP2-D)¶
Implements ForgePort. REST for contents/refs/PRs/labels/commit-status,
GraphQL only for enablePullRequestAutoMerge (the one mutation with no REST
equivalent). commit_files uses the Git Data API (create tree from
base_sha's tree + files, create commit, update ref with
force=False — GitHub itself then supplies the "ref moved" 422/409 that
this adapter converts to TransientError, matching ports.py's documented
contract). open_or_update_pull_request is idempotent per branch — GitHub's
"list PRs for this branch" REST call first, create only if none exists,
otherwise return the existing number unchanged (never edits title/body on
the update path unless they actually differ, to avoid a no-op PR-edit event
storm).
11. adapters/local_files.py / adapters/system_clock.py (WP2-G)¶
local_files.py: every method resolves path against a fixed root
(constructor argument, e.g. the repo checkout root) via Path(root, path).resolve()
and raises ValidationError if the resolved path is not .is_relative_to(root)
(catches both ..-traversal and absolute-path attempts in one check, per
ports.py's documented contract) before touching the filesystem.
list_files(prefix) uses Path.rglob("*") filtered to files, returned as
/-joined POSIX-style relative strings (not OS-native os.sep) so output is
stable across platforms and matches InMemoryFiles' fake behavior exactly.
system_clock.py: datetime.now(UTC).strftime("%Y-%m-%dT%H:%M:%SZ") — one
line, no configuration surface.
12. cli/*.py (WP2-H..M)¶
Each subcommand module exposes one function matching cli/main.py's
existing _DISPATCH shape: def run(args: argparse.Namespace) -> ExitCode.
Registration (_DISPATCH["reconcile"] = reconcile.run, plus the matching
subparsers.add_parser(...) args) is WP2-M's production-wiring job, done
last, once every subcommand module exists — do not edit cli/main.py's
_build_parser/_DISPATCH from an individual WP2-H..L work package; land
your module's run function and its own tests, leave wiring to WP2-M.
cli/announce.py— removed (0.5.0,adr_forge_neutral_owners.mdD3). It was a second implementation of the wire-format writer, in a second language, on no production path: publishing isocx package announce's job (ocx-sh/ocx), which is whatocx-mirroractually shells out to. Its subcommand, its wiring entry (_run_announce,_index_forge) and its docs went with it.core/did not:core/regenerate.py,core/observe.pyandcore/verify_claims.pyare the read/verify half and are shared byreconcile,validateandseed-import.cli/reconcile.py(rewritten verify-only, fork-PR announce revamp, 2026-07-18 — owner-confirmed decision set "Verify-only reconcile"):FilePort.list_files("p/")to enumerate every*.jsonroot (excluding CAS subtrees, unchanged glob rule) -> per package,core/verify_claims.py::verify_claimsre-derives every claimed tag/desc blob from registry truth, pluscore/anomaly.py::check_tag_mutations(reused verbatim) for pinned-tag mutation detection. Never writes top/at all — no regenerate, no diff, no commit, no PR; the--dry-runflag this module used to carry is gone entirely (verify-only is always "dry"). Escalation (which findings raiseAnomalyError, exit 65):check_tag_mutations's pinned-tag mutations always escalate;verify_claims's"cas-object-*"/"desc-blob-*"findings always escalate (structural CAS integrity, independent of tag semantics — the same unconditional treatmentcheck_no_dangling_references/check_digest_self_consistentalready give);"digest-mismatch"does not escalate on its own (floating-tag drift is expected cascade behavior, ADR-1 D2/D3, and that same digest mismatch on a pinned tag is already caught by the reusedcheck_tag_mutations— avoids double-flagging one phenomenon two ways)."tag-missing-upstream"does escalate (ADR-6 FP-2/FP-3 — a decided rule, no longer an open question) unless the committedTagEntry.yanked is not Nonefor that tag: yank is grace, an explicit owner-authorized exemption from the registry-existence check —_PackageReportcarries the committed root's yanked-tag names precisely so_escalating_findingscan tell a yanked-and-vanished tag apart from a plain silent drop. A non-empty escalating-finding set opens/updates one anomaly issue viaForgePort.create_or_update_issue(promoted onto the port this stage, see §3/§10) before raising.cli/validate.py(extended, fork-PR announce revamp — byte-exact discipline): takes changed-file paths as CLI positional args (unchanged). New, before the existing structural gauntlet: parse the committed root bytes, re-serialize viavalidate_entry.serialize_package_root, and byte-compare against the committed bytes — a mismatch isValidationError, "committed bytes are not the canonical root serialization" (CI re-derives the PR's own claimed canonical form; a publisher's tooling is expected to already emit exactly this form). Also new: every committed CAS file under the package'so/sha256/tree (tag objects and desc readme/logo blobs alike, not only the ones a tag/desc field references) is hash-checked against its own filename-declared digest viavalidate_entry.check_digest_self_consistent— closes the gap where only referenced tag objects were ever byte-verified. Unless--offline: in addition to the existingcheck_digest_in_scope/check_ownershipG-15 checks, wirescore/verify_claims.py::verify_claims— any finding ->ValidationError(a claim about registry content that isn't true right now is this PR's problem, not an anomaly against committed history; contrastcli/reconcile.py's disposition for the identical finding taxonomy)."mismatch"or anyValidationError-> exit"unconfirmed"-> print a WARN to stderr, exit 0 (ADR-4 Risk 2, unchanged). Also takes--base-dir DIR(optional): the directory the PR gate materializes each changed root's BASE-ref bytes into. ADR-2 ND-4 gates claiming a reserved segment, not updating a root already committed under one — so whencheck_namespace_not_reservedrejects a segment, the rejection is retracted iff (a) the same path exists at the base ref and (b)core/diff.classify_change(base, head) == "refresh", and even then only for the deployment's ownreserved_namespaces(the exact set--allow-reserved-namespaceopens;ALWAYS_RESERVED_SEGMENTS— control-path and generic — is never admitted by any amount of base-ref state). Without--base-dir, or with no such root at the base ref, every reserved segment is a fresh claim — fail-closed. This is what makesocx package announce --forkusable for the operator's own first-party roots: that command can open nothing but FORK PRs, and a fork PR never receives--allow-reserved-namespace. Repointingrepository, editingowners[], flippingstatus/deprecated_message, or moving a yank marker are all outside"refresh", so they stay rejected by this REQUIRED check rather than merely routed to the human lane behind the non-requiredgovernance/review-requiredstatus.cli/render.py(reshaped byplan_site_redesignWP-bot):FilePort.list_files("p/")under--index-dir-> parse every root + every CAS object intoSourcePackage->render.build_render_plan-> write the returned file tuple under--out.--outisrequired=Trueat the argparse layer (cli/main.py's_add_render_argumentsowns the missing-flag usage error, so this module never raises its ownValidationErrorfor it);--site-dist(the old wrapper-pages target) is gone — this CLI subcommand emits exactly one output tree now, so there is no second invocation or--phasesplit. This subcommand does not itself invoke the VitePress build — that'srender-deploy.yml's job (task render:build), which runssite:buildfirst (dynamic routes globp/*/*.jsondirectly, no wrapper-page pre-emission needed) andindexbot render --outsecond, into the sameemptyOutDir-wiped tree.--checkcomputes the plan and reports drift against--outwithout writing,ExitCode.VALIDATION_FAILUREon drift.cli/seed_import.py: reads localCATALOG.md(title/description/ keywords — frontmatter shape TBD by whoever writes this WP; note the precedent inocx_lib::package::description::Frontmatter, §7's desc.py section, if useful) +logo.svg/logo.png+mirror.ymlviaFilePort, thenobserveagainst the live registry to build the initialtagsmap. Open question / dependency gap:mirror.ymlimplies YAML parsing;bot/pyproject.tomlhas no YAML dependency (httpxis the only runtime dep, per BD-1's minimal-footprint driver) and this stage may not editpyproject.toml. Flag the missingpyyaml/ruamel.yamldev-or-runtime dependency in this WP'sopen_questionsrather than adding it unilaterally — or parsemirror.ymlwith a deliberately tiny hand-rolledkey: valuereader if its real shape turns out to be that simple (confirm shape against actual seed data before choosing).cli/classify_pr.py:ForgePort.get_pull_request_info(pr_number)(from--pr-numberCLI arg) -> for each.changed_pathsentry matching a root path shape,get_file_contents(path, info.base_sha)andget_file_contents(path, info.head_sha), parse each (missing base file ->None, matchingdiff.classify_change'sbefore: PackageRoot | None) ->diff.classify_change-> the worst classification across all changed roots wins ("human-review-required">"new-package">"refresh"— a PR touching two packages where one is a refresh and one needs human review is human-review-required overall) ->add_labels.cli/governance_check.py(extended, fork-PR announce revamp — G-19/ G-20): re-derives the classification viaclassify_pr.classify_pull_request(unchanged single-source-of-truth approach), then: machine lane (refresh) requires the PR author's numeric forge id (PullRequestInfo.author_id, §3) to appear inowners[]of every touched package root, read from the base ref only (never the PR head — the samegovernance-gatetrust boundarycli/classify_pr.pyalready documents) — pass ->success; fail -> falls back to the human lane below. Human lane (new-package/human-review-required, or a refresh PR that failed G-19): alwayspending+ reviewers assigned from committed.github/maintainers.yml(parsed viacore/maintainers.py::parse_maintainers, read from the base ref) minus the PR author (self-review carve-out — GitHub's API itself rejects assigning a PR's own author as their own reviewer) viaForgePort.request_reviewers, plus one idempotent comment viaForgePort.create_comment(hidden HTML marker<!-- indexbot:governance -->— update-in-place on repeated runs, never reposted). Neverfailure— nothing has actually gone wrong, the PR just needs a human. (ADR-4 BD-5's fuller "green for refresh PRs onceschema-validateis also green" cross-job condition remains deferred, per the original entry this replaces — unaffected by G-19/G-20.) Writes the resulting commit-status state ("success"/"pending") to$GITHUB_OUTPUTasdisposition—.github/workflows/governance.yml'sarm-auto-mergejob runsindexbot governance-gate --arm-only --disposition <that value>, and arms strictly onsuccess, never on the rawclassify-prlabel (announce-revamp Phase 3 — a label-based check cannot see the G-19 ownership result). Anything else, the empty string a FAILED gate job publishes included, withdraws instead: that job runs onif: ${{ !cancelled() }}so a gate that errored cannot leave a pull request armed on an evaluation that never finished.
13. Consolidated open questions carried into Phase 2¶
- Yank-on-republish (
core/regenerate.py, §7): does a re-published digest under a yanked tag name clear the yank? Default: no. Confirm before Phase 3. - Silent tag disappearance — resolved, ADR-6 FP-2/FP-3 (fork-PR
announce revamp, 2026-07-18): a tag vanishing from the registry
(
core/verify_claims.py's"tag-missing-upstream"finding) escalates to an anomaly incli/reconcile.py's verify-only sweep unless the committedTagEntry.yanked is not Nonefor that tag — yank is grace, an explicit owner-authorized exemption; everything else vanished-upstream is an anomaly, never a silent drop. No longer decided bycore/regenerate.py(which never runs in the verify-only sweep at all any more, §12). - Pinned-vs-floating anomaly predicate (
core/anomaly.py, §7) — resolved, 2026-07-29: a tag is pinned iff it carries a build fragment (3.28.1_20260216, prefix and prerelease included). The rolling cascade targets3.28.1/3.28/3/latestare exempt. The stated default ("exactX.Y.Zonly is pinned") was the exact inverse of OCX's cascade semantics and shipped that way — both failure modes this item warned about were live simultaneously on the nightly sweep. See §7. - Issue-creation on
ForgePort(cli/reconcile.py, §12) — resolved, fork-PR announce revamp 2026-07-18:create_or_update_issuepromoted ontoports.ForgePort(§3/§10), implemented inGitHubApi(unchanged body — it already existed as an adapter-only capability) andFakeGitHub.cli/reconcile.py's verify-only sweep now calls it directly on a non-empty escalating-finding set, before raisingAnomalyError. mirror.ymlparsing (cli/seed_import.py, §12): implies a YAML dependency not currently declared; this stage may not editpyproject.toml.governance-check's cross-job read ofschema-validate's result (cli/governance_check.py, §12): default proposal is a workflow-levelneeds:/if:gate rather than an API poll from inside the CLI; confirm this is sufficient when WP2-S designsvalidate.yml's actual job graph.
14. Root serializer — client-facing byte-exact spec¶
Added fork-PR announce revamp, 2026-07-18, to give the byte-exact discipline
(§12's cli/validate.py entry) a single, precise, standalone reference —
this is the spec ocx#216 (the client-side port of this same
serialization, for a publisher tool implemented outside this repo) ports
against. Restates §5/§1 in one place rather than requiring a cross-reader to
reassemble it from two sections; not a new rule — validate_entry.py's
serialize_package_root remains the one authoritative implementation of the
root form, and this section documents its exact output byte-for-byte.
p/<namespace>/<package>.json (the package root — human-diffable, PR-review
form, never digested itself):
- UTF-8 encoded,
ensure_ascii=True(thejson.dumpsdefault, not passed explicitly but never overridden either) — non-ASCII field values (e.g. a non-ASCIIdeprecated_message,desc.title/.description, orupstream.disclaimer) serialize as\uXXXXescapes, never raw UTF-8 bytes. Aserde_json(or any other) port that defaults toto_string_pretty's UTF-8-passthrough behavior instead will byte-diverge on the first non-ASCII value it serializes — required reading for ocx#216. json.dumps(data, indent=2, sort_keys=False)— 2-space indent, insertion-order preserved (never alphabetized).- Key order is fixed and matches
model.PackageRoot's declared field order exactly:name,repository,owners,status,deprecated_message,created,desc,upstream(omitted entirely whenNone— schema forbidsnullthere),superseded_by(omitted entirely whenNone, identical omit-when-absent contract),source(same contract),variants(same contract, and omitted when empty, never[]— the schema'sminItems: 1refuses the other spelling, which is what keeps every root predating the field byte-identical so no announce rewrites it),tags— always last. Nested objects (owners[],desc,tags[*],tags[*].yanked) use their own dataclass's declared field order the same way — seevalidate_entry.py's_*_to_dicthelpers for the exact per-type key list. owners[]carries four keys per entry, in this order:login,id,github,github_id(adr_forge_neutral_owners.mdD1/D2).login/idare canonical and forge-neutral —loginis a forge username, never a display name, becauseForgePort.request_reviewershands it straight to the forge (GitLab'snameis a different field and resolves to nobody);idis the numeric forge user id and is the ownership key G-19 matches on.github/github_idare the pre-0.5.0 spelling, emitted derived from the canonical pair —model.Ownercannot express them separately. The read side takeslogin/idwhen present and falls back togithub/github_id, so every root published before 0.5.0 parses unchanged, and refuses a root carrying both spellings in disagreement: that would show one identity to a human reviewer and hand another to the auto-merge gate. Dropping the legacy pair is a breaking change gated onformat_version, not a later quiet cleanup.desc: Noneserializes as the JSON literalnull(the key itself is never omitted — this is the one field whose absence-vs-null semantics differ fromupstream/superseded_byabove).upstream's own three fields are mixed optional semantics, not a single rule applied uniformly:orgis always present (required, non-nullable);repository_urlis omitted entirely whenNone(schema forbidsnullthere, matchingupstreamitself);disclaimeris always present, serialized as the JSON literalnullwhenNone(schema allowsnullthere — the one sub-field ofupstreamthat behaves likedescabove rather than likerepository_url).- A single trailing
\n— always present, exactly one byte, no more. - Byte-exact discipline (this revamp): CI re-derives this exact form from
the PR's own parsed root (
parse_package_root->serialize_package_root) and byte-compares against the committed bytes. A root that parses correctly but isn't already in this exact canonical form (different key order, different indent, minified, missing/extra trailing newline, ...) is rejected —cli/validate.py's "committed bytes are not the canonical root serialization" failure. A publisher's own tooling (ocx package announce, or a third-party port of this spec) must emit exactly this form, not merely schema-equivalent JSON.
p/<namespace>/<package>/o/sha256/<hex>.json (the content-addressed CAS
object):
- Not serialized by this bot. It is the exact byte sequence the physical
registry returned for
GET /v2/<repo>/manifests/<tag>, stored unmodified;<hex>issha256of those bytes, which is the registry's own manifest digest for that image index. There is no canonical form to re-derive, so the byte-exact rules above have no counterpart here — CI verifies the hash and the document kind (cli/validate.py, §12:parse_image_index_digestsrejects anything that is not an OCI image index), never a re-serialization. This resolves ADR OQ3: theo/gate is a kind check, not a round-trip. - Bounded at 4 MiB (
core/observe.py's_MAX_INDEX_BYTES). Verbatim storage hands the publisher's registry control of how many bytes each tag commits — an image index'sannotationsare unbounded — soobserve_one_tagrefuses anything larger withValidationError. 4 MiB is the OCI distribution spec's manifest size, not a number this repo invented. - Whitespace, key order and
manifests[]order are the registry's. Two tags resolving to byte-identical index responses still dedup to one object (ADR-1 D3) — the registry's own digest already guarantees it. - Desc blobs (
o/sha256/<hex>.md— readme,o/sha256/<hex>.{svg,png}— logo) follow the same rule for the same reason: copied verbatim from the physical registry's__ocx.descartifact layers, digest =sha256of those exact bytes.
tests/golden/serializer/ (WP-P0-P1, 2026-07-24) holds this section's
committed byte vectors for the root form — real serialize_package_root
output, never hand-typed — and tests/core/test_serializer_golden.py is the
gate that rides task bot:test.
15. core/policy.py — deployment policy (.github/index-policy.json)¶
G-03's registry-host allowlist is a per-deployment input, not a constant.
OCX's index is one format, many copies: the public ocx-sh/index serves
bytes from ghcr.io, a corporate copy from its own Harbor/Artifactory/ECR.
Each index repo commits its own policy:
parse_index_policy(raw: bytes) -> IndexPolicy is that file's whole
grammar — name and name_segments are required with no defaults, since an
index that does not declare its own identity would publish under another
deployment's. The optional keys (reserved_namespaces, governance, ci)
and the full grammar are in Deployment policy. ValidationError on anything else: malformed JSON, a non-object
document, an unknown key (a typo'd registry_host would otherwise leave a
deployment with no policy while looking like it had one), a missing or
non-array registry_hosts, an empty array, or an entry that is not a bare
lowercase host. The host shape is strict because the alternative is silent:
check_repository_allowlisted matches against urlsplit().hostname, which is
always lowercased and never carries a port, so https://harbor.corp,
Harbor.Corp and harbor.corp:5000 would each parse fine and then match
nothing. A registry on a non-default port is allowlisted by its bare host
(harbor.corp admits oci://harbor.corp:5000/team/tool).
A committed file, never an environment or Actions variable. "Extend only
via reviewed PR" is G-03's control — repository is the pointer every ocx
client follows to fetch bytes, so widening the allowlist is a supply-chain
trust decision. A repo/Actions variable can be changed by anyone with settings
access, silently, with no diff and no reviewer; a committed file under
.github/** keeps widening mechanically equal to a reviewed PR, on the same
surface branch protection and CODEOWNERS already guard, next to this repo's
other bot-read governance data (maintainers.yml, G-20).
No JSON Schema of its own. schema/*.schema.json is the served wire
contract ($id: https://index.ocx.sh/schema/...), sealed by
adr_locked_observation_index_format.md D7; this file is never served and is
not part of the index format. parse_index_policy is its single source of
truth, runs in CI on every indexbot invocation that needs a policy, and
tests/security/test_governance_contracts.py parses the committed file
itself (test_g03_shipped_policy_is_exactly_ghcr_io — the public index's
effective policy stays exactly {"ghcr.io"}, and a PR that widens the shipped
file fails there).
Where it is loaded, and the no-adapter guard¶
cli/_wiring.py — the composition root, the only module that constructs
adapters — loads the policy at wiring time, before the subcommand does any
work, and passes the resulting frozenset[str] into reconcile.run,
validate.run and seed_import.run as a keyword-only
allowed_hosts. render/classify-pr/governance-check never resolve a
repository and deliberately need no policy file at all (the same
per-subcommand independence that already governs env-var requirements there).
Source of the bytes: the local checkout via FilePort for
validate/reconcile/seed-import; the base ref via ForgePort for the
privileged subcommands, which never check the repository out.
Two failures are raised there, both loud and both early:
- No policy file — fail closed. An index copy that never stated a policy
says so, rather than silently inheriting the public index's
ghcr.io. - A host no
RegistryPortadapter can serve — the important one.adapters/registry_v2.pyserves the hosts_wiring._registry()wires a client for, and nothing else. Allowlistingharbor.corp.internaltoday would therefore produce a root that passes every validation check and then cannot be fetched — strictly worse than the honest refusal it replaces._wiring.REGISTRY_ADAPTER_HOSTSis the honest statement of what is implementable,_registry_hostsrefuses any policy that exceeds it, and the error names the missing piece (implement aRegistryPort, add its host, dispatch it).RoutedRegistryrepeats the refusal at call time for a host with no client — unreachable behind the policy check, kept as the backstop for a future wiring bug.
Why PR-head validation is not a bypass¶
validate.yml's unprivileged schema-validate job runs indexbot validate
against PR-head content by design — it checks the PR's own claims, and holds
no credential. It therefore also loads the PR's own .github/index-policy.json.
That is not a self-authorization hole:
- The policy path is outside every root's refresh scope, so a PR touching it
is classified human-lane by
cli/classify_pr.pyand can never auto-merge (ADR-6 FP-5 — asserted in_OUT_OF_SCOPE_PATHS). Merging a widened policy requires a human, which is precisely the control. - Today the no-adapter guard closes it outright anyway: the only servable host
is
ghcr.io, so a PR-head policy naming anything else fails the run.
announce is the one flow that deliberately does not read a local policy:
its publisher runs outside any index checkout, so it reads the target index's
committed policy at main over the API instead (§12).