Research A — How package-index / repository-metadata formats evolve their schema

Q1 — Tagged vs untagged unions

04There is no general discriminator mechanism. Variant-ish fields are stringly-typed tags read leniently. The dependency kind is a plain optional string:

05

/// The dependency kind. "dev", "build", and "normal". pub kind: Option<Cow<'a, str>>,

06— crates/cargo-util-schemas/src/index.rs, accessed 2026-08-09.

07The book states the same as a defaulting rule, not a closed enum:

08

"The dependency kind. "dev", "build", or "normal". If not specified or null, it defaults to "normal"."

09— https://doc.rust-lang.org/cargo/reference/registry-index.html, accessed 2026-08-09.

10The recorded trouble case is features2, which is precisely an untagged-variant problem. The features map's values silently gained new syntax (dep:, pkg?/feat) with no tag, so old readers mis-parsed them. The fix was to move the new-shaped values into a separate field and add a schema tag (v):

11

"This is separated from features because versions older than 1.19 will fail to load due to not being able to parse the new syntax, even with a Cargo.lock file."

12— crates/cargo-util-schemas/src/index.rs, accessed 2026-08-09.

13This is the single most instructive data point in the whole subject: an in-place widening of a value's grammar, without a tag, was a breaking change; the repair was to split the field and version the entry.

Q2 — Absent vs empty vs null

14Documented per-field defaults, and — notably — the defaults were added to the docs only in Cargo 1.84, i.e. the rule was under-specified for years:

15

"Array of features (as strings) enabled for this dependency. Since Cargo 1.84, defaults to [] if not specified."

16

"Set of features defined for the package. Each feature maps to an array of features or dependencies it enables. Since Cargo 1.84, defaults to {} if not specified."

17

"Boolean of whether or not this is an optional dependency. Since Cargo 1.84, defaults to false if not specified."

18

"Boolean of whether or not default features are enabled. Since Cargo 1.84, defaults to true if not specified."

19— https://doc.rust-lang.org/cargo/reference/registry-index.html, accessed 2026-08-09.

20null is explicitly given the same meaning as absent, per field:

21

"The target platform for the dependency. If not specified or null, it is not a target dependency."

22

"The URL of the index of the registry where this dependency is from as a string. If not specified or null, it is assumed the dependency is in the current registry."

23— same page.

24So: absent == null == the documented default. "Empty" and "not recorded" are NOT distinguished in this format. The reader enforces it with serde defaults:

25

#[serde(default)] pub features: BTreeMap<Cow<'a, str>, Vec<Cow<'a, str>>>,

26

#[serde(default = "default_true")] pub default_features: bool,

27— crates/cargo-util-schemas/src/index.rs, accessed 2026-08-09.

28One field carries the archaeology of an absent-vs-empty mistake being tolerated forever:

29

/// If true, Cargo will skip this version when resolving. /// /// This was added in 2014. Everything in the crates.io index has this set /// now, so this probably doesn't need to be an option anymore. pub yanked: Option<bool>,

30— same file, accessed 2026-08-09. The field is de facto always present but remains Option because the type can never be narrowed once published.

Q3 — Closed vocabularies

31NOT FOUND: any documented "unknown value" fallback or extension convention for enum-like values in the crates.io index. Searched: the Cargo Book registry-index page (full text, keywords unknown, reserved, extension), cargo-util-schemas/src/index.rs, RFC 2789, RFC 3143. What exists instead is that enum-ish values are not modelled as enums at all — kind is Option<Cow<str>>, so an unknown value survives deserialization. The format's answer to "new kind of thing" is the v schema counter (Q5), not per-value tolerance.

Q4 — Strictness (ignore vs reject unknown fields)

32There is no normative statement in the spec. The rule is implementation-defined and lenient: the reader struct carries no deny_unknown_fields, so serde's default — ignore unknown keys — applies.

33Verified mechanically: grep -c "deny_unknown_fields" crates/cargo-util-schemas/src/index.rs → 0 (accessed 2026-08-09).

34The nearest thing to a normative statement is the forward-compatibility paragraph that opens the format definition:

35

"New features are occasionally added, which are only understood starting with the version of Cargo that introduced them. Older versions of Cargo may not be able to use packages that make use of new features. However, the format for older packages should not change, so older versions of Cargo should be able to use them."

36— https://doc.rust-lang.org/cargo/reference/registry-index.html, accessed 2026-08-09.

37Asymmetry: writers are constrained, readers are not. See Q7.

Q5 — Version field semantics — the v field

38This is the best-documented reader contract found in any subject for at-rest data.

39

"An unsigned 32-bit integer value indicating the schema version of this entry. If this is not specified, it should be interpreted as the default of 1. Cargo (starting with version 1.51) will ignore versions it does not recognize. This provides a method to safely introduce changes to index entries and allow older versions of cargo to ignore newer entries it doesn't understand. Versions older than 1.51 ignore this field, and thus may misinterpret the meaning of the index entry. The current values are: 1: The schema as documented here, not including newer additions. This is honored in Rust version 1.51 and newer. 2: The addition of the features2 field. This is honored in Rust version 1.60 and newer."

40— https://doc.rust-lang.org/cargo/reference/registry-index.html, accessed 2026-08-09.

41The reader contract is therefore: higher-than-known v ⇒ skip THIS ENTRY (this package version), not the file, and not an error. The unit of version-gating is the record, not the document. This is the key structural choice, and it is enabled by the physical layout — the index is newline-delimited JSON, one record per line:

42

"The rest of the index repository contains one file for each package, where the filename is the name of the package in lowercase. Each version of the package has a separate line in the file."

43— https://doc.rust-lang.org/cargo/reference/registry-index.html, accessed 2026-08-09.

44NDJSON + per-record version means an unparseable or too-new record costs you exactly that one package version, while every other version of the same package remains resolvable. A single JSON document with a document-level version would make the same change cost the whole package.

45The source adds a v3 not yet in the book, and is franker about the cost:

46

/// The schema version for this entry. /// /// If this is None, it defaults to version 1. Entries with unknown /// versions are ignored. /// /// Version 2 schema adds the features2 field. /// /// Version 3 schema adds artifact, bindep_targes, and lib for /// artifact dependencies support. /// /// This provides a method to safely introduce changes to index entries /// and allow older versions of cargo to ignore newer entries it doesn't /// understand. This is honored as of 1.51, so unfortunately older /// versions will ignore it, and potentially misinterpret version 2 and /// newer entries. /// /// The intent is that versions older than 1.51 will work with a /// pre-existing Cargo.lock, but they may not correctly process cargo /// update or build a lock from scratch. In that case, cargo may /// incorrectly select a new package that uses a new index schema. A /// workaround is to downgrade any packages that are incompatible with the /// --precise flag of cargo update.

47— crates/cargo-util-schemas/src/index.rs, accessed 2026-08-09. (Note bindep_targes — a typo preserved in the shipped doc comment.)

48No major/minor split. v is a single monotonic integer.

49Crucially: adding a field does NOT bump v. links (2018), rust_version (2023) and pubtime (2025) were all added as plain optional fields with no v bump; only changes that would make an old reader misinterpret existing data got a bump:

50

/// Native library name this package links to. /// /// Added early 2018 (see <https://github.com/rust-lang/cargo/pull/4978>), /// can be None if published before then. pub links: Option<Cow<'a, str>>,

51

/// Corresponds to package.rust-version. /// /// Added in 2023 (see <https://github.com/rust-lang/crates.io/pull/6267>), /// can be None if published before then or if not set in the manifest. pub rust_version: Option<RustVersion>,

52— same file, accessed 2026-08-09.

53The RFC states the design intent explicitly:

54

"The version field is added to help prevent older versions of Cargo from updating to newer versions of package that it doesn't understand. Cargo, since 1.51, already supports the "v" field, and will ignore any entries with a "v" value greater than 1."

55— RFC 3143, accessed 2026-08-09.

56And the PR that introduced it:

57

"This adds a v field to the index which indicates a format version for an index entry. If Cargo encounters a version newer than it understands, it will ignore those entries. This makes it safer to make changes to the index entries (such as adding new things), and not need to worry about how older cargos will react to it. In particular, this will make it safer to run cargo update on older versions if we ever decide to add new things to the index. Currently this is not written anywhere, and is intended as a safety guard for the future. For now I will leave it undocumented until we actually decide to start using it."

58— ehuss, PR #9161 body, commented 2021-02-10, merged by bors 2021-02-22T16:26:28Z; https://github.com/rust-lang/cargo/pull/9161, accessed 2026-08-09. (Verified verbatim against the raw page HTML, not via summarisation.)

59This is the most actionable sentence in the entire research. The version field was shipped undocumented and unused, purely as a forward-compatibility guard, four years (v1 → the first real use at v2) before it was needed. That ordering is forced: a version gate only protects you if it is already deployed in readers before the first change that needs it. Cargo's own docs record what it costs when it is not — "Versions older than 1.51 ignore this field, and thus may misinterpret the meaning of the index entry." The readers that predate the guard can never be protected retroactively.

60config.json has no version field at all — its documented keys are only dl, api, auth-required (https://doc.rust-lang.org/cargo/reference/registry-index.html, accessed 2026-08-09). The version discipline was applied to records, never to the index-level config.

Q6 — Field identity and reuse

61Fields are identified by JSON key name. NOT FOUND: any reserved-name list, or any stated policy on renaming/removing/reusing a field name, in the Cargo Book or cargo-util-schemas. Searched the registry-index page full text for reserved, rename, remove, deprecat; the only reserved hit concerns package names ("Rejects reserved names, such as Windows special filenames like 'nul'"), not fields.

62What is documented instead is that the same concept carries different names in three sibling formats, and the doc warns readers about it:

63

"Note: The index JSON format has subtle differences from the JSON format of the Publish API and cargo metadata. If you are using one of those as a source to generate index entries, you are encouraged to carefully inspect the documentation differences between them."

64

"req — The Publish API field is called version_req." "vers — The cargo metadata field is called version." "default_features — The cargo metadata field is called uses_default_features."

65— https://doc.rust-lang.org/cargo/reference/registry-index.html, accessed 2026-08-09.

66Worse, the same field name means different things across the three:

67

"registry — cargo metadata uses a value of null to indicate that the dependency comes from crates.io. The index uses a value of null to indicate that the dependency comes from the same registry as the index."

68

"name — When the dependency is renamed in Cargo.toml, the publish API puts the original package name in the name field and the aliased name in the explicit_name_in_toml field. The index places the aliased name in the name field, and the original package name in the package field."

69— same page. This is an observed cost of name reuse across formats, documented as a hazard rather than prevented by policy.

Q7 — Round-trip preservation

70The rule is stated for writers, and it is the strongest immutability rule found anywhere in this research:

71

"The JSON objects should not be modified after they are added except for the yanked field whose value may change at any time."

72— https://doc.rust-lang.org/cargo/reference/registry-index.html, accessed 2026-08-09.

73That is not "preserve unknown fields on rewrite" — it is "do not rewrite at all". The index is append-only per record, with exactly one mutable field. Round-trip preservation is achieved by forbidding round-trips.

Q8 — Migration mechanism

74Policy is never break, only add, plus per-record version-gating for the cases where adding is not enough:

75

"However, the format for older packages should not change, so older versions of Cargo should be able to use them."

76— https://doc.rust-lang.org/cargo/reference/registry-index.html, accessed 2026-08-09.

77The transport migration (git index → sparse HTTP index) was handled entirely outside the document schema, by a URL-scheme change (sparse+https://) — the per-file JSON payload was kept identical. RFC 2789, accessed 2026-08-09.

78Regret note on the mechanism itself: it only works for readers ≥ 1.51, and the design document says so plainly — see §5.

Q1 — Tagged vs untagged unions

81OCI is the strongest "always tag" case found. Every reference to external content is a descriptor, and the type tag is REQUIRED on the descriptor:

82

"- mediaType string This REQUIRED property contains the media type of the referenced content. Values MUST comply with [RFC 6838][rfc6838], including the [naming requirements in its section 4.2][rfc6838-s4.2]."

83— descriptor.md, accessed 2026-08-09.

84And descriptors are designed so that sibling entries may legitimately differ in type:

85

"When other formats contain multiple descriptors, unless otherwise specified, those descriptors are independent of each other, allowing fields like the mediaType and the algorithm for the digest to vary within that external content."

86— descriptor.md, accessed 2026-08-09.

87Why mandatory: so a reader can decide whether to parse before fetching, and can refuse to parse safely:

88

"Implementations MUST NOT attempt to parse the referenced content if this media type is unknown and instead consider the referenced content as arbitrary binary data (e.g.: as application/octet-stream)."

89— manifest.md, accessed 2026-08-09.

90The recorded trouble case is the in-band tag being optional. In the manifest itself (as opposed to the descriptor pointing at it), mediaType is only SHOULD:

91

"- mediaType string This property SHOULD be used and remain compatible with earlier versions of this specification and with other similar external formats. When used, this field MUST contain the media type application/vnd.oci.image.manifest.v1+json. This field usage differs from the descriptor use of mediaType."

92— manifest.md (main), accessed 2026-08-09.

93That wording is itself the result of a reversal. At tags v1.0.0 and v1.0.1 the field was not recommended at all — it was merely parked:

94

"- mediaType string This property is reserved for use, to maintain compatibility. When used, this field contains the media type of this document, which differs from the descriptor use of mediaType."

95— manifest.md @ v1.0.0 and @ v1.0.1, both verified 2026-08-09. The promotion from reserved to SHOULD be used landed in v1.0.2 (verified by diffing the three tags on 2026-08-09).

96So OCI ended up with a self-describing tag at the reference site (mandatory from the start) and a tag inside the document that took three point releases to go from "reserved" to "recommended" — and still is not required. Readers that fetch a bare document without its descriptor have to sniff. The lesson is directional: the in-band type tag kept getting stronger, never weaker.

Q2 — Absent vs empty vs null

97OCI's answer is unusual and directly relevant: rather than allow absent, it invented an explicit empty value so that a required field can be present-but-meaningless:

98

"In the scenarios were an artifact does not have content for the config or any layers, an empty JSON descriptor is recommended as a placeholder to satisfy the requirement that these fields exist, while serving as an indication to consuming tools that there is no useful content in that descriptor."

99— OCI blog 2024-03-13, accessed 2026-08-09. (The sentinel is application/vnd.oci.empty.v1+json, a descriptor whose content is the two bytes {}.)

100That is: "empty" was made a first-class tagged value rather than an absence, precisely because absence would have been ambiguous to older readers that require the field.

101descriptor.md also constrains when a reader may add data:

102

"Implementations MUST NOT populate the data field in situations where doing so would modify existing content identifiers. For example, a registry MUST NOT arbitrarily populate data fields within uploaded manifests, as that would modify the content identifier of those manifests."

103— descriptor.md, accessed 2026-08-09.

104For the one open-ended map in the format, the rule is stated normatively — and it explicitly makes absent and empty equivalent, permitting either:

105

"- If there are no annotations then this property MUST either be absent or be an empty map."

106

"- While the value MUST be present, it MAY be an empty string."

107— annotations.md, accessed 2026-08-09.

108So OCI's answer to Q2 is: do not distinguish absent from empty — mandate that they mean the same thing, and let writers choose. There is no "unknown/not recorded" third state.

109NOT FOUND: a general normative rule in image-spec on omitting vs emitting []/{} for array fields such as layers. Searched descriptor.md, manifest.md, image-index.md, considerations.md for empty, omit, null.

Q3 — Closed vocabularies

110OCI states an unknown-value tolerance rule repeatedly and normatively, at every place a vocabulary appears. This is the densest such evidence in the whole research.

111For layer media types:

112

"Implementations storing or copying image manifests MUST NOT error on encountering a mediaType that is unknown to the implementation."

113For config media types:

114

"Implementations storing or copying image manifests MUST NOT error on encountering a value that is unknown to the implementation."

115For artifact types:

116

"Implementations storing or copying image manifests MUST NOT error on encountering an artifactType that is unknown to the implementation."

117— all manifest.md, accessed 2026-08-09.

118For index entries:

119

"Future versions of the spec MAY use a different mediatype (i.e. a new versioned format). An encountered mediaType that is unknown to the implementation MUST NOT generate an error."

120— image-index.md, accessed 2026-08-09.

121Checksum algorithms are the model case for extending a closed set. The algorithm is a prefix tag inside the value, the grammar is open, and unknown algorithms MUST pass syntactic validation:

122

"Implementations SHOULD allow digests with unrecognized algorithms to pass validation if they comply with the above grammar. While sha256 will only use hex encoded digests, separators in algorithm and alphanumerics in encoded are included to allow for extensions. As an example, we can parameterize the encoding and algorithm as multihash+base58:QmRZxt2b1FVZPNqd8hsiykDL3TdBDeTSPX9Kv46HmX4Gx8, which would be considered valid but unregistered by this specification."

123— descriptor.md, accessed 2026-08-09.

124With a registry and a mandatory floor:

125

"The following algorithm identifiers are currently defined by this specification: | sha256 | SHA-256 | | sha512 | SHA-512 | | blake3 | BLAKE3 | If a useful algorithm is not included in the above table, it SHOULD be submitted to this specification for registration."

126

"Implementations MUST implement SHA-256 digest verification for use in descriptors."

127— descriptor.md, accessed 2026-08-09. (BLAKE3 is a later addition to that table — the mechanism has actually been exercised.)

Q4 — Strictness

128Normative, and it was WEAKENED between v1.0.2 and v1.1.0 — see §5.

129Current (main, and v1.1.0 / v1.1.1):

130

"Implementations processing content SHOULD NOT generate an error if they encounter an unknown property in a known media type."

131— considerations.md (main), accessed 2026-08-09.

132Former (identical text at tags v1.0.0 and v1.0.2, verified 2026-08-09):

133

"Implementations that are reading/processing manifests or image indexes MUST NOT generate an error if they encounter an unknown property. Instead they MUST ignore unknown properties."

134— considerations.md @ v1.0.2.

135Asymmetric by role, not by writer/reader. v1.1 split one rule into two, addressed at two different kinds of implementation:

136

"Implementations storing or copying content MUST NOT modify or alter the content in a way that would change the digest of the content. Examples of these implementations include: - A [registry implementing the distribution specification][distribution-spec], including local registries, caching proxies - An application which copies content to disk or between registries Implementations processing content SHOULD NOT generate an error if they encounter an unknown property in a known media type. Examples of these implementations include: - A [runtime implementing the runtime specification][runtime-spec] - An implementation using OCI to retrieve and utilize artifacts, e.g.: a WASM runtime"

137— considerations.md (main), accessed 2026-08-09.

138Note the qualifier added in v1.1: "in a known media type". Tolerance of unknown properties is conditional on recognising the type. That is the tag doing load-bearing work: you only forgive what you can locate.

139And the tolerance rule is stronger for the designated extension namespace than for the schema proper. Unknown annotation keys are still a MUST NOT, never downgraded:

140

"- Consumers MUST NOT generate an error if they encounter an unknown annotation key."

141— annotations.md, accessed 2026-08-09.

142Contrast with the schema-level rule quoted above, which is now only SHOULD NOT. OCI therefore offers a two-tier extensibility contract: put speculative data in the open, reverse-DNS-namespaced annotation map and you get a hard guarantee that readers will not choke; invent a new top-level property and you get only a soft one.

143Compliance is defined only over MUST-class terms:

144

"An implementation is not compliant if it fails to satisfy one or more of the MUST, MUST NOT, REQUIRED, SHALL, or SHALL NOT requirements for the protocols it implements."

145— spec.md, accessed 2026-08-09. Which means the v1.0→v1.1 downgrade to SHOULD NOT moved unknown-property tolerance out of the compliance boundary.

Q5 — Version field semantics — schemaVersion is vestigial

146

"- schemaVersion int This REQUIRED property specifies the image manifest schema version. For this version of the specification, this MUST be 2 to ensure backward compatibility with older versions of Docker. The value of this field will not change. This field MAY be removed in a future version of the specification."

147— manifest.md (main), accessed 2026-08-09; the identical sentence is already present at tag v1.0.2, verified 2026-08-09.

148This is the sharpest negative result in the research: OCI has a schema-version field, it is REQUIRED, it is frozen at a constant inherited from a predecessor format (Docker manifest schema 2), it carries no reader contract at all, and the spec announces it may be deleted. Versioning is done by media type instead — application/vnd.oci.image.manifest.v1+json — i.e. the version lives in the type tag, not in a numeric field.

Q6 — Field identity and reuse

149Fields are identified by name; there is an explicit Reserved policy for extension fields:

150

"### Reserved Extended Descriptor field additions proposed in other OCI specifications SHOULD first be considered for addition into this specification."

151— descriptor.md, accessed 2026-08-09.

152Annotations are the sanctioned open namespace, and its naming policy is fully normative — this is the most complete field-identity regime found in any subject:

153

"- Keys MUST be unique within this map, and best practice is to namespace the keys. - Keys SHOULD be named using a reverse domain notation - e.g. com.example.myKey. - The prefix org.opencontainers is reserved for keys defined in Open Container Initiative (OCI) specifications and MUST NOT be used by other specifications and extensions. - Keys using the org.opencontainers.image namespace are reserved for use in the OCI Image Specification and MUST NOT be used by other specifications and extensions, including other OCI specifications."

154— annotations.md, accessed 2026-08-09. Note the second reservation: OCI reserved a sub-namespace against its own sibling specifications, not only against third parties.

155The rules are binding wherever annotations appear:

156

"This OPTIONAL property MUST use the annotation rules."

157— descriptor.md, accessed 2026-08-09.

158NOT FOUND: any policy explicitly forbidding renaming or re-purposing an existing schema field name. Searched descriptor.md, manifest.md, image-index.md, considerations.md, spec.md for rename, reuse, repurpose, deprecat.

Q7 — Round-trip preservation — mandatory, and enforced by hashing

159

"Implementations storing or copying content MUST NOT modify or alter the content in a way that would change the digest of the content."

160— considerations.md (main), accessed 2026-08-09. This clause did not exist in v1.0.2 (verified by diffing the two tags on 2026-08-09) — it was added in v1.1.0.

161The blog restates it as a design assumption of the whole migration:

162

"To facilitate this transition period, changes to the image spec added new fields which should be ignored by registries without generating errors or modifying the content of the manifest."

163— OCI blog 2024-03-13, accessed 2026-08-09.

164Preservation is not merely required, it is structurally unavoidable: the document's identity is its digest, so any dropped field yields a different object that no longer matches the descriptor pointing at it. Content-addressing converts "please preserve unknown fields" from a policy into a physical law.

Q8 — Migration mechanism

165Additive, with the media type as the escape hatch of last resort:

166

"Future versions of the spec MAY use a different mediatype (i.e. a new versioned format)."

167— image-index.md, accessed 2026-08-09.

168And an out-of-band extension namespace on the distribution side:

169

"Extensions to the registry API may be added by implementations without requiring a change to the OCI distribution specification. This allows development and testing of new features, in addition to custom features in a registry. Extensions are denoted by a leading _ in their API path, which is invalid for a repository name."

170

"To prevent conflicts, extensions should register their name with the distribution specification."

171— OCI blog 2024-03-13, accessed 2026-08-09.

172Plus an explicit deprecation channel:

173

"Registries may now include a warning header in responses. This can be used to notify users of issues that are not yet error conditions, including deprecation announcements."

174— OCI blog 2024-03-13, accessed 2026-08-09.

175The v1.1 migration also shipped a capability-probe + fallback pattern rather than a version bump, because old readers could not be upgraded in time:

176

"Clients pushing a manifest with a subject field and querying for referrers have an additional responsibility to manage and use a fallback tagged Index when the registry does not support the referrers API. When a registry supports the referrers API, it will return the header OCI-Subject set to the digest in the subject field in response to a client pushing a manifest with the subject field. Similarly, when querying for referrers, a registry that supports the API will always return a 200 response, even if the response is an empty Index, to distinguish between registries that have not implemented the API and may respond with a 404 or similar error."

177— OCI blog 2024-03-13, accessed 2026-08-09. And the honest cost accounting:

178

"The client requirement to manage the fallback tag is subject to race conditions since the existing content of the fallback tag must first be queried to add the new entry. The fallback tags also clutter the tag listing with entries that are not the images end users are searching for."

179— same source.

Q1 — Tagged vs untagged unions

182No per-field type tag. Variant selection is out-of-band — it depends on which kind of control file you are reading. This is stated normatively:

183

"The presence and purpose of a field, and the syntax of its value, may differ between types of control files."

184— Policy §5.1, https://www.debian.org/doc/debian-policy/ch-controlfields.html, accessed 2026-08-09 (verbatim in deb822(5) too).

185The consequences are exactly the untagged-union failure mode. The same field name carries different arity and different separators depending on context:

186

"This field contains a list of files with information about each one. The exact information and syntax varies with the context." (Files — 3 columns in .dsc, 5 in .changes)

187

"This folded field is a list of binary packages. Its syntax and meaning varies depending on the control file in which it appears." … "When it appears in a .changes file, it lists the names of the binary packages being uploaded, separated by whitespace (not commas)."

188

"When used inside a .changes file, the Description field has a different format than in source or binary control files."

189— Policy §5.6.21, §5.6.19, §5.6.13, accessed 2026-08-09.

190And Architecture is a four-way untagged union discriminated only by inspecting the word:

191

"A unique single word identifying a Debian machine architecture … An architecture wildcard identifying a set of Debian machine architectures … all, which indicates an architecture-independent package. … source, which indicates a source package."

192— Policy §5.6.8, accessed 2026-08-09.

193Recorded trouble cases:

194(a) Format collided with itself — the same name means two unrelated things, and Policy had to say so out loud when it finally documented both:

195

"The Format field of .changes files is now 1.8. The Format field syntax for source package .dsc files allows a subtype in parentheses, and it is used for a different purpose than the Format field for .changes files."

196— Policy upgrading-checklist §10.33, Version 3.9.0, released June 2010, https://www.debian.org/doc/debian-policy/upgrading-checklist.html, accessed 2026-08-09.

197(b) Rules-Requires-Root had to have an in-band discriminator retrofitted so two variants could share one value slot:

198

"A space separated list of keywords described below. These keywords must always contain a forward slash, which sets them apart from the other possible values of Rules-Requires-Root."

199— Policy §5.6.31, accessed 2026-08-09.

200(c) One field's shape is discriminated by the presence of a different field:

201

"As an exception to Policy 5.6.13 (Description), the value of the Description field may omit the long description if the Description-md5 field is defined."

202— DebianRepository/Format wiki, https://wiki.debian.org/DebianRepository/Format?action=raw, accessed 2026-08-09.

203(d) apt silently degrades an unrecognised variant rather than failing:

204

c _error->Warning("Unknown Multi-Arch type '%.*s' for package '%s'", (int)MultiArch.size(), MultiArch.data(), Package().c_str()); MA = pkgCache::Version::No;

205— apt apt-pkg/deb/deblistparser.cc, debListParser::ParseMultiArch, https://salsa.debian.org/apt-team/apt/-/raw/main/apt-pkg/deb/deblistparser.cc, accessed 2026-08-09.

Q2 — Absent vs empty vs null

206There is no null, "empty" is largely illegal, and empty is NOT distinguishable from "not recorded". Omission is the canonical encoding of both "no data" and "the default".

207

"Empty field values are only permitted in source package template control files (debian/control). Such fields are ignored."

208— Policy §5.1, accessed 2026-08-09.

209Writers are normatively told to drop them:

210

"These tools are responsible for removing the line breaks from such fields when using fields from debian/control to generate other control files. They are also responsible for discarding empty fields."

211— Policy §5.2, accessed 2026-08-09.

212An empty list means omit the field, stated separately for two fields:

213

"If no binary packages are being uploaded, this field will not be present."

214— Policy §5.6.19 (Binary) and §5.6.13 (Description in .changes), accessed 2026-08-09.

215In one case, absence is the only legal way to express the default — the format cannot say the default out loud:

216

"Note that, due to limitations in the archive management software, this value cannot currently be specified explicitly in binary package control files. Instead, the field needs to be absent in order to imply its default value and debhelper discards it when transforming a source package control template."

217— Policy §5.6.30 (Multi-Arch: no), accessed 2026-08-09.

218

"In source package template control files, the Package-Type field should be omitted instead of giving it a value of deb, as this value is assumed for stanzas lacking this field."

219— Policy §5.6.28, accessed 2026-08-09.

220Confirmed in dpkg: an empty value is a silent no-op, indistinguishable from absence —

221

c f_priority(...) { const char *str = value; int priority; if (!*value) return;

222— dpkg lib/dpkg/fields.c, https://salsa.debian.org/dpkg-team/dpkg/-/raw/main/lib/dpkg/fields.c, accessed 2026-08-09.

Q3 — Closed vocabularies

223The governing rule Debian arrived at: a new algorithm becomes a new field; a new enum value is only safe where an unknown-fallback was designed in from the start.

224Checksums — new algorithm = new field, never a new value in an old field:

225

"These multiline fields contain a list of files with a checksum and size for each one. Both Checksums-Sha1 and Checksums-Sha256 have the same syntax and differ only in the checksum algorithm used"

226— Policy §5.6.24, accessed 2026-08-09. The legacy MD5 Files field was never removed and is still mandatory alongside:

227

"The list of files in these fields must match the list of files in the Files field."

228— Policy §5.6.24, accessed 2026-08-09.

229Weak algorithms are retired by a reader rule instead of by deletion:

230

"Clients may not use the MD5Sum and SHA1 fields for security purposes, and must require a SHA256 or a SHA512 field."

231— DebianRepository/Format wiki, accessed 2026-08-09.

232Unknown Checksums-* fields are silently not read at all, because dpkg iterates its own hardcoded algorithm table rather than the file's fields:

233

perl my $CHECKSUMS = { md5 => { name => 'MD5', … strong => 0, }, sha1 => { name => 'SHA-1', … strong => 0, }, sha256 => { name => 'SHA-256', … strong => 1, }, }; perl sub add_from_control { … foreach my $alg (checksums_get_list()) { my $key = "Checksums-$alg";

234— dpkg scripts/Dpkg/Checksums.pm, https://salsa.debian.org/dpkg-team/dpkg/-/raw/main/scripts/Dpkg/Checksums.pm, accessed 2026-08-09. (No SHA-512 in dpkg's table.)

235Where an unknown-fallback WAS designed in, it works. Priority has an explicit catch-all that preserves the unrecognised value verbatim:

236

"The known priorities are required, important, standard, optional, extra, and unknown, but other values can be used as well."

237— deb-control(5), dpkg-dev 1.23.7, https://manpages.debian.org/unstable/dpkg-dev/deb-control.5.en.html, accessed 2026-08-09.

238

c priority = parse_nv(ps, PARSE_NV_FALLBACK, &str, priorityinfos); … if (str == NULL) { pkg->priority = PKG_PRIO_OTHER; pkg->otherpriority = nfstrsave(value);

239— dpkg lib/dpkg/fields.c, accessed 2026-08-09.

240Package types and architecture names are likewise open:

241

"Simple field containing a word indicating the type of package: deb for binary packages and udeb for micro binary packages. Other types not defined here may be indicated."

242— Policy §5.6.28, accessed 2026-08-09.

243

"/ Ignore unknown types for forward-compatibility. /"

244— dpkg lib/dpkg/arch.c, https://salsa.debian.org/dpkg-team/dpkg/-/raw/main/lib/dpkg/arch.c, accessed 2026-08-09.

245The best-designed part of the format is extension *inside values* — namespaced keywords with a MUST-ignore rule, and positional lists declared open-ended:

246

"All tools must ignore keywords under namespaces they do not know or own. A tool may emit a warning, or abort with an error, if it finds unknown keywords in namespaces it owns, but it is not required to do this for all keywords in the namespace."

247— Policy §5.6.31, accessed 2026-08-09.

248

"Other space-separated keyword=value items may be introduced in the future, and users of this field must ignore items with unknown keywords."

249— Policy §5.6.33 (Git-Tag-Info), accessed 2026-08-09.

250

"Fifth and subsequent space-separated items may be present and parsers must allow them."

251— Policy §5.6.27 (Package-List), accessed 2026-08-09.

252Counter-example — a new value in an existing field DID break old readers:

253

"APT 1.8 added automatic subkey support and support for disabling it via ! as used by gnupg. As APT before 1.8 will choke on this new syntax it should be avoided until all expected users have a sufficiently recent client."

254— DebianRepository/Format wiki (Signed-By), accessed 2026-08-09.

Q4 — Strictness

255NOT FOUND: Debian Policy contains no general normative statement about what a reader must do with an unknown field. Searched every Policy chapter source (ch-controlfields.rst, ch-archive.rst, ch-source.rst, ch-binary.rst, ch-relationships.rst, ch-customized-programs.rst, upgrading-checklist) for unknown field, unrecognised, unrecognized, must ignore, should ignore. The only hits concern unknown keywords inside a value (quoted in Q3), not unknown fields.

256Policy §5.7 is a writer rule, not a reader rule — and it is the format's designated extension mechanism:

257

"Additional user-defined fields may be added to the source package template control file. Such fields will be ignored, and not copied to (for example) binary or Debian source control files or Debian upload changes control files. If you wish to add additional unsupported fields to these output files you should use the mechanism described here. Fields in the source package template control file with names starting X, followed by one or more of the letters BCS and a hyphen -, will be copied to the output files. Only the part of the field name after the hyphen will be used in the output file. Where the letter B is used the field will appear in binary package control files, where the letter S is used in Debian source control files and where C is used in Debian upload changes control files."

258— Policy §5.7, https://www.debian.org/doc/debian-policy/ch-controlfields.html#user-defined-fields, accessed 2026-08-09.

259The de-facto rule is "tolerate", and it differs per tool — the same unknown field is retained by one tool and dropped with a warning by another:

• 260dpkg's database parser silently retains unknown fields (arbs list); it only errors if the name is under 2 characters or duplicated — dpkg lib/dpkg/parse.c, pkg_parse_field(), https://salsa.debian.org/dpkg-team/dpkg/-/raw/main/lib/dpkg/parse.c, accessed 2026-08-09.
• dpkg's control-file transformer warns and drops:

261

perl } elsif (not field_is_allowed_in($field, $from_type)) { warning(g_("unknown information field '%s' in input data in %s"), $field, $from->get_option('name') || g_('control information')); }

262— dpkg scripts/Dpkg/Control/FieldsCore.pm, field_transfer_single(), accessed 2026-08-09.

• 263apt's tag parser retains unknown tags in a secondary hash bucket (Key::Unknown → BetaIndexes) so they stay retrievable — apt apt-pkg/tagfile.cc, accessed 2026-08-09.

264Index readers are explicitly warned to expect undocumented fields:

265

"Note that the control file of .deb files may contain additional fields not yet documented by policy or not yet documented here which then might also be found in this file."

266— DebianRepository/Format wiki, accessed 2026-08-09.

267The X- convention has an acknowledged flaw: the prefix is stripped on output, so user-defined fields land in the global namespace:

268

"Take into account that these user-defined fields will be using the global namespace, which might at some point in the future collide with officially recognized fields. To avoid such potential situation you can prefix those fields with Private-, such as XB-Private-New-Field."

269— deb-src-control(5), dpkg-dev 1.23.7, https://manpages.debian.org/unstable/dpkg-dev/deb-src-control.5.en.html, accessed 2026-08-09.

Q5 — Version field semantics

270There is no format version on a Packages, Sources, or Release file. The Release file's Version: is the distribution release number, not a schema version:

271

"The Version field, if specified, shall be the version of the release. This is usually a sequence of integers separated by the character . (full stop)."

272— DebianRepository/Format wiki, accessed 2026-08-09.

273Four different version-ish things exist, none interchangeable:

1. 274Standards-Version — package↔Policy conformance, explicitly not parse-affecting:

275

"The version number has four components: major and minor version number and major and minor patch level. When the standards change in a way that requires every package to change the major number will be changed."

276— Policy §5.6.11, accessed 2026-08-09.

277

"For a package to have an old Standards-Version value is not itself a bug, however. It just means that no-one has yet reviewed the package with changes to the standards in mind."

278— Policy §3.1.1, accessed 2026-08-09.

1. 279Format in .changes — a real, semver-shaped schema version with a stated compatibility rule:

280

"The value of this field declares the format version of the file. The syntax of the field value is a version number with a major and minor component. Backward incompatible changes to the format will bump the major version, and backward compatible changes (such as field additions) will bump the minor version. The current format version is 1.8."

281— deb-changes(5), dpkg-dev 1.23.7, https://manpages.debian.org/unstable/dpkg-dev/deb-changes.5.en.html, accessed 2026-08-09.

282NOT FOUND: any statement of what a reader MUST do on a higher .changes Format. The major/minor writing rule is stated; the corresponding reading rule is not.

1. 283Format in .dsc — a dispatch key, hard-rejected if the major is unknown, and the minor is ignored and overwritten:

284

perl my ($major, $minor, $variant) = $self->{format}->set($format); my $module = "Dpkg::Source::Package::V$major"; … if ($@) { error(g_("source package format '%s' is not supported: %s"), $format, $@); }

285— dpkg scripts/Dpkg/Source/Package.pm, https://salsa.debian.org/dpkg-team/dpkg/-/raw/main/scripts/Dpkg/Source/Package.pm, accessed 2026-08-09.

1. 286dpkg-build-api — a behaviour API level declared via a build-dependency; the newest and most deliberate versioning mechanism in the ecosystem:

287

"The source package dpkg build API level, defines a versioned interface for source packages, where each API level provides specific behaviors and guarantees. These interfaces can then be adopted by packages in a gradual way, and phased out more easily than with global behavior changes."

288— dpkg-build-api(7), https://manpages.debian.org/unstable/dpkg-dev/dpkg-build-api.7.en.html, accessed 2026-08-09.

Q6 — Field identity and reuse

289Case-insensitive on read; canonical mixed case on write — and the index format tightens this into an explicit reader/writer asymmetry:

290

"Field names are not case-sensitive, but it is usual to capitalize the field names using mixed case as shown below. Field values are case-sensitive unless the description of the field says otherwise."

291— Policy §5.1, accessed 2026-08-09 (verbatim in deb822(5)).

292

"In addition to the rules for control files, field names shall be generated using the case defined in this document, that is, code creating repositories shall be case-sensitive, but code reading repositories should not be case-sensitive."

293— DebianRepository/Format wiki, accessed 2026-08-09. This is Postel's law stated as a normative rule about a single syntactic property, and it is one of only two places in this research where writer-strict/reader-lax is spelled out as policy (the other is Go's Parse/ParseLax).

294Name syntax and uniqueness:

295

"The field name is composed of US-ASCII characters excluding control characters, space, and colon (i.e., characters in the ranges U+0021 (!) through U+0039 (9), and U+003B (;) through U+007E (~), inclusive). Field names must not begin with the comment character (U+0023 #), nor with the hyphen character (U+002D -)."

296— Policy §5.1, accessed 2026-08-09.

297

"5.1 A control stanza must not contain more than one instance of a particular field name."

298— upgrading-checklist §10.33, Version 3.9.0 (June 2010), accessed 2026-08-09. Note that this elementary constraint was only written down in 2010, ~15 years into the format's life.

299Renaming / removing / reusing a field name: NOT FOUND as stated policy. No reserved-name list exists in Policy. What exists is de-facto practice in dpkg: renamed fields keep a permanent read-side alias that warns and is never written back (writer function w_null):

300

c /* The following are the obsolete fields that get remapped to their * modern forms, while emitting an obsolescence warning. */ { FIELD("Recommended"), f_obs_dependency, w_null, dep_recommends }, { FIELD("Optional"), f_obs_dependency, w_null, dep_suggests }, { FIELD("Class"), f_obs_class, w_null }, { FIELD("Revision"), f_obs_revision, w_null }, { FIELD("Package-Revision"), f_obs_revision, w_null }, { FIELD("Package_Revision"), f_obs_revision, w_null },

301— dpkg lib/dpkg/parse.c, accessed 2026-08-09. Still shipping today: Recommended→Recommends, Optional→Suggests, Class→Priority. Read forever, write never is the de-facto rename protocol.

Q7 — Round-trip preservation

302NOT FOUND: no normative MUST/SHOULD anywhere — Policy, deb822(5), deb-control(5), or the repository-format wiki — requiring a rewriting tool to preserve fields it did not understand.

303dpkg's actual behaviour is split three ways:

1. 304The status/available database DOES round-trip unknown fields — parsed into arbs, written back after all known fields:

305

c for (fip = fieldinfos; fip->name; fip++) { fip->wcall(vb, pkg, pkgbin, fw_printheader, fip); } for (afp = pkgbin->arbs; afp; afp = afp->next) { varbuf_add_arbfield(vb, afp, fw_printheader); }

306— dpkg lib/dpkg/dump.c, varbuf_stanza(), https://salsa.debian.org/dpkg-team/dpkg/-/raw/main/lib/dpkg/dump.c, accessed 2026-08-09. Content is preserved; relative field order is not — unknown fields are appended at the end.

1. 307Obsolete (known-but-retired) fields are deliberately NOT round-tripped (w_null, Q6).

1. 308Control-file generators drop unknown fields with a warning unless X[SBC]--prefixed (Q4).

309The Perl deb822 library layer does preserve, and documents it:

310

"The order in which fields have been set is remembered and is used to be able to dump back the same content. The output order can also be overridden if needed. You can store arbitrary values in the hash, they will always be properly escaped in the output to conform to the syntax of control files."

311— dpkg scripts/Dpkg/Control/HashCore.pm, https://salsa.debian.org/dpkg-team/dpkg/-/raw/main/scripts/Dpkg/Control/HashCore.pm, accessed 2026-08-09.

Q8 — Migration mechanism

312Dominant strategy: additive, "never break, only add". No format-wide migration mechanism exists for Packages/Sources/Release. Decades of pure accretion, from Policy's own upgrading-checklist (https://www.debian.org/doc/debian-policy/upgrading-checklist.html, accessed 2026-08-09):

314Old readers are an explicit design constraint, with graceful degradation prescribed instead of versioning:

315

"Any tool (particularly older versions of them) may be unaware of this field and behave like the field was set to binary-targets. The package build must gracefully cope with this and produce a semantically equivalent result."

316— Policy §5.6.31, accessed 2026-08-09.

317When a break was genuinely needed, five different mechanisms were used — there is no single one: (1) versioned dispatch key with hard rejection (.dsc Format); (2) opt-in build-dependency-declared API level (dpkg-build-api); (3) a temporary field designed to be deleted, purely to sequence a transition —

318

"An optional field with a temporary and very specific usecase … The field's purpose is to decouple the introduction of indexes like Contents-all from the introduction of Packages-all. Support by clients (if they choose to support it at all) is therefore bound to disappear after the transition to all for all indexes is done."

319— DebianRepository/Format wiki (No-Support-for-Architecture-all), accessed 2026-08-09; (4) dated deprecation with a removal floor (one-line sources.list, §5 R-D1); (5) permanent read-side aliasing (Q6).

320Removals happen, but on ~decade timescales: Origin and Bugs were added in dpkg 1.7.0 (2000-11-05) and removed in dpkg 1.19.5 (2019-02-23) — dpkg debian/changelog, https://salsa.debian.org/dpkg-team/dpkg/-/raw/main/debian/changelog, accessed 2026-08-09.

321A sixth mechanism, and the most transferable one: migrate by changing the FILENAME, so old readers never open the new format at all. apt's deb822 sources live in files with a new extension, and pre-1.1 readers simply skip them:

322

"Files in this format have the extension .sources."

323

"This is a new format supported by apt itself since version 1.1. Previous versions ignore such files with a notice message as described earlier."

324— sources.list(5), apt 3.3.2, https://manpages.debian.org/unstable/apt/sources.list.5.en.html, accessed 2026-08-09 (independently re-fetched and verified).

325Old and new coexist in the same directory; the reader's file-extension filter is the version check. No in-band version field is needed, and there is no ambiguity for a reader that understands only one of them.

326Debian also has a delta convention for list-valued fields, which lets a new field extend a default without restating it:

327

"Multivalue fields like Architectures also have Architectures-Add and Architectures-Remove to modify the default value rather than replacing it."

328— same source, accessed 2026-08-09. (The one-line format has the equivalent -= / += operators.) This is a direct answer to "how do I let a consumer adjust a list I own without either of us clobbering the other".

329And a documented instance of extending a format breaking third-party parsers — the exact risk profile of an index read by third-party tools:

330

"Note that not all options as described below are supported by all apt versions. Note also that some older applications parsing this format on their own might not expect to encounter options as they were uncommon before the introduction of multi-architecture support."

331— same source, accessed 2026-08-09.

Q1 — Tagged vs untagged unions

333go.mod is a verb-tagged line format: every line's first token is its discriminator (module, go, require, exclude, replace, retract, tool, ignore, godebug, toolchain). The grammar is stated as such:

334

"GoMod = { Directive } . Directive = ModuleDirective | GoDirective | ToolDirective | IgnoreDirective | RequireDirective | ExcludeDirective | ReplaceDirective | RetractDirective ."

335— https://go.dev/ref/mod, accessed 2026-08-09.

336go.sum hashes are algorithm-tagged inside the value, with a pre-declared upgrade path:

337

"The hash column consists of an algorithm name (like h1) and a base64-encoded cryptographic hash, separated by a colon (:). Currently, SHA-256 (h1) is the only supported hash algorithm. If a vulnerability in SHA-256 is discovered in the future, support will be added for another algorithm (named h2 and so on)."

338— https://go.dev/ref/mod, accessed 2026-08-09.

339The parser even keeps its own recogniser deliberately loose so that future syntax tweaks stay detectable:

340

// for forward compatibility. // (This code has to be work to identify new toolchains even if we tweak the syntax in the future.) var ToolchainRE = lazyregexp.New(^default$|^go1($|\.))

341— modfile/rule.go, accessed 2026-08-09 (typo "has to be work" is in the source).

Q2 — Absent vs empty vs null

342For the proxy .info document, one field is required and one optional, with absence given explicit meaning:

343

"The Version field is required and must contain a valid, canonical version (see Versions). The $version in the request path does not need to be the same version or even a valid version; this endpoint may be used to find versions for branch names or revision identifiers. However, if $version is a canonical version with a major version compatible with $module, the Version field in a successful response must be the same. The Time field is optional. If present, it must be a string in RFC 3339 format."

344— https://go.dev/ref/mod, accessed 2026-08-09.

345For .mod, "absent upstream" is normalised into a synthesised minimal document rather than an error or an empty response:

346

"If the module does not have a go.mod file at the requested version, a file containing only a module statement with the requested module path must be returned."

347— same source.

348NOT FOUND: a general rule about emitting []/{} vs omitting collections in the GOPROXY JSON. Searched https://go.dev/ref/mod full text for empty, omit, null.

Q3 — Closed vocabularies

349Two mechanisms, both explicit:

1. 350Hash algorithms: a named-prefix scheme with a reserved successor (h1 → h2), quoted in Q1.
2. go.mod directives: unknown verbs are tolerated in dependencies, rejected in your own module — see Q4. That is a role-scoped fallback rather than a value-level one.

Q4 — Strictness — the clearest asymmetry found anywhere

351Go parses the same file format under two different strictness modes depending on whose file it is. Strict for the main module (unknown directive is an error), lax for dependencies (unknown directive is silently skipped) — and the reason given is explicitly forward compatibility:

352

// ParseLax is like Parse but ignores unknown statements. // It is used when parsing go.mod files other than the main module, // under the theory that most statement types we add in the future will // only apply in the main module, like exclude and replace, // and so we get better gradual deployments if old go commands // simply ignore those statements when found in go.mod files // in dependencies.

353— modfile/rule.go, accessed 2026-08-09.

354

// The [Parse] and [ParseLax] functions both parse a go.mod file and return an // abstract syntax tree. ParseLax ignores unknown statements and may be used to // parse go.mod files that may have been developed with newer versions of Go.

355— modfile/rule.go package doc, accessed 2026-08-09.

356The implementation, with the allow-list of directives that matter even in a dependency:

357

go func (f *File) add(errs *ErrorList, block *LineBlock, line *Line, verb string, args []string, fix VersionFixer, strict bool) { // If strict is false, this module is a dependency. // We ignore all unknown directives as well as main-module-only // directives like replace and exclude. It will work better for // forward compatibility if we can depend on modules that have unknown // statements (presumed relevant only when acting as the main module) // and simply ignore those statements. if !strict { switch verb { case "go", "module", "retract", "require", "ignore": // want these even for dependency go.mods default: return } }

358and in strict mode:

359

go switch verb { default: errorf("unknown directive: %s", verb)

360— modfile/rule.go, accessed 2026-08-09.

361The rule of thumb this encodes: be strict about data you are the authority for; be lax about data you merely consume. Not writer-vs-reader — owner-vs-consumer.

Q5 — Version field semantics — the go directive, and a real tightening

362

"A go directive indicates that a module was written assuming the semantics of a given version of Go. The version must be a valid Go version, such as 1.14, 1.21rc1, or 1.23.0. The go directive sets the minimum version of Go required to use this module. Before Go 1.21, the directive was advisory only; now it is a mandatory requirement: Go toolchains refuse to use modules declaring newer Go versions."

363— https://go.dev/ref/mod, accessed 2026-08-09.

364So the reader contract on a higher version changed from "warn and try" to "refuse". The toolchain document is more explicit, and records that the new gate was backported into already-released readers:

365

"Before Go 1.21, Go toolchains treated the go line as an advisory requirement: if builds succeeded the toolchain assumed everything worked, and if not it printed a note about the potential version mismatch. Go 1.21 changed the go line to be a mandatory requirement instead. This behavior is partly backported to earlier language versions: Go 1.19 releases starting at Go 1.19.13 and Go 1.20 releases starting at Go 1.20.8, refuse to load workspaces or modules declaring version Go 1.22 or later."

366— https://go.dev/doc/toolchain, accessed 2026-08-09.

367That backport is the answer to the problem crates.io documented but could not fix (pre-1.51 Cargo silently misinterpreting v): Go shipped the version check into patch releases of old readers, so that old readers would refuse rather than misread. The tightening was then made survivable by pairing it with automatic toolchain download:

368

"When the go or toolchain line is newer than the bundled toolchain, the go command runs the newer toolchain instead. For example, when using the go command bundled with Go 1.21.3 in a main module that says go 1.21.9, the go command finds and runs Go 1.21.9 instead. It first looks in the PATH for a program named go1.21.9 and otherwise downloads and caches a copy of the Go 1.21.9 toolchain. This automatic toolchain switching can be disabled, but in that case, for more precise forwards compatibility, the go command will refuse to run in a main module or workspace in which the go line requires a newer version of Go."

369— https://go.dev/doc/toolchain, accessed 2026-08-09.

370The blunt statement of the rule:

371

"The Go toolchain refuses to load a module or workspace that declares a minimum required Go version greater than the toolchain's own version. For example, Go 1.21.2 will refuse to load a module or workspace with a go 1.21.3 or go 1.22 line."

372— same source. For go.work, the same contract is stated as version-directed interpretation:

373

"The go directive indicates the go toolchain version with which the go.work file is intended to work. If changes are made to the go.work file format, future versions of the toolchain will interpret the file according to its indicated version."

374— https://go.dev/ref/mod, accessed 2026-08-09.

375The GOPROXY protocol itself has no version number. NOT FOUND: any protocol version, negotiation, or capability header for GOPROXY. Searched https://go.dev/ref/mod for "protocol version", "version of the protocol" (0 hits each; "GOPROXY protocol" occurs 11 times, always as the name of the endpoint set). The protocol is versioned only by the fact that new endpoints (e.g. @latest) are declared optional:

376

"This endpoint is optional, and module proxies are not required to implement it."

377— https://go.dev/ref/mod, accessed 2026-08-09.

Q6 — Field identity and reuse — an explicit reservation, and the best single quote in the research

378

"More fields may be added in the future, so other names are reserved."

379— https://go.dev/ref/mod, GOPROXY $base/$module/@v/$version.info description, accessed 2026-08-09.

380This is the only place in any subject surveyed where a format pre-emptively reserves the entire unused name space of a JSON object. It converts "unknown field" from an extensibility question into a conflict-avoidance rule binding third parties: you may not invent your own key in that object, because the name might later be spent by the spec.

Q7 — Round-trip preservation — mandatory, verbatim, for proxies

381

"Otherwise, the original, unmodified go.mod file must be returned."

382— https://go.dev/ref/mod, $base/$module/@v/$version.mod, accessed 2026-08-09.

383A proxy is a byte-preserving relay, not a re-serializer. Backed by hashing:

384

"After downloading a .mod or .zip file, the go command computes a cryptographic hash and checks that it matches a hash in the main module's go.sum file."

385— same source. Again: preservation enforced by content-addressing, not by good manners.

386Note the deliberate exception — the two mutable endpoints are explicitly declassified:

387

"Note that version lists and version metadata returned for .info requests are not authenticated and may change over time."

388— same source.

Q8 — Migration mechanism

389Additive-plus-lax-readers (Q4), reserved names (Q6), a pre-registered successor for the one crypto value that must eventually rotate (Q1). The one genuinely breaking change — the go directive becoming mandatory — was shipped with an automatic remediation (toolchain switching), not with a migration guide. See §5.

Q1 — Tagged vs untagged unions — the worst case found

392npm's metadata is full of untagged unions, documented as such, because the packument copies user-authored package.json shapes through verbatim. The funding field is the clearest, with three shapes and no discriminator:

393

"funding: object containing a URL that provides up-to-date information about ways to help fund development of your package, or a string URL, or an array of these"

394— docs/responses/package-metadata.md, accessed 2026-08-09.

395npm's own docs confirm the same shorthand-or-object pattern elsewhere:

396

"If you have a single executable, and its name should be the name of the package, then you can just supply it as a string." (on bin)

397

"For GitHub, GitHub gist, Bitbucket, or GitLab repositories you can use the same shortcut syntax you use for npm install" (on repository)

398— https://docs.npmjs.com/cli/v11/configuring-npm/package-json/, accessed 2026-08-09.

399The recorded trouble case is license. Historically it was an untagged union of string, object, and a sibling plural field licenses holding an array of objects. It was deprecated in favour of a single string form:

400

"Some old packages used license objects or a "licenses" property containing an array of license objects"

401

"Those styles are now deprecated. Instead, use SPDX expressions"

402— https://docs.npmjs.com/cli/v11/configuring-npm/package-json/, accessed 2026-08-09.

403The escape hatch for values outside the closed SPDX vocabulary is itself a string sentinel, not a new shape:

404

"If you are using a license that hasn't been assigned an SPDX identifier, or if you are using a custom license, use a string value like this one: { "license": "SEE LICENSE IN <filename>" }"

405— same page. (See §5 for the deprecation timeline.)

Q2 — Absent vs empty vs null — the only explicit tri-state found

406The abbreviated packument states a general absence rule:

407

"The name, version, and dist fields will always be present. The others will be absent if they are irrelevant for this package version."

408— docs/responses/package-metadata.md, accessed 2026-08-09.

409and then, for exactly one field, distinguishes false from unknown:

410

"_hasShrinkwrap: true if this version is known to have a shrinkwrap that must be used to install it; false if this version is known not to have a shrinkwrap. If this field is undefined, the client must determine through other means if a shrinkwrap exists."

411— same source, accessed 2026-08-09.

412That is a genuine three-valued field: true / false / not recorded, with a documented reader obligation in the third case. It is the only such contract found in any subject.

Q3 — Closed vocabularies

413Checksums are the live case, and npm solved it by adding a new tagged field beside the old untagged one rather than changing the old field's meaning:

414

"- shasum: the SHA-1 sum of the tarball - integrity: since Apr 2017, string in the format <hashAlgorithm>-<base64-hash>, refer the [Subresource Integrity] and [cacache] package for more"

415and, listed as a future item in the same list:

416

"- (in the future) a SHA-2 512 sum of the tarball"

417— docs/responses/package-metadata.md, accessed 2026-08-09.

418shasum is permanently SHA-1 because its name and its untagged value make it unchangeable; integrity carries the algorithm inside the value (SRI form), so it can rotate. Note also the doc's habit of stamping each field with the date it appeared — since Apr 2017, since Feb 2018, since Apr 2018 — an informal per-field version marker in lieu of a schema version.

419NOT FOUND: any documented fallback rule for an unrecognised enum value in the packument. Searched both npm registry docs files for unknown, unrecognized, MUST, SHOULD.

Q4 — Strictness

420NOT FOUND: any normative statement (MUST/SHOULD) that readers ignore or reject unknown fields. Searched docs/responses/package-metadata.md and docs/REGISTRY-API.md for MUST, SHOULD, unknown, ignore, additional — the documents contain no RFC-2119 language at all.

421What is documented is that the full packument is an open object carrying arbitrary publisher-controlled keys, which makes reader leniency mandatory in practice:

422

"The full version object will also contain any other fields the package publisher chose to include in their package.json file for that version."

423— docs/responses/package-metadata.md, accessed 2026-08-09.

424The abbreviated form is the opposite — a closed allow-list:

425

"This form of the package metadata exists to provide a smaller payload designed to support installation. It contains an allow list of fields from the full metadata set."

426— same source, accessed 2026-08-09.

427So npm runs both policies at once on the same data: an open document for humans/tools, a curated closed projection for installers. The two are selected by content negotiation (Q5).

428A candid admission that no validation was ever applied on the way in:

429

"Historically no validation has been performed on those fields; they are generated by parsing user-provided data in package.json at publication time."

430— docs/responses/package-metadata.md (on human objects), accessed 2026-08-09.

431The documentation has visibly lost the race with the format — one field is documented as:

432

"- directories:???"

433— docs/REGISTRY-API.md, accessed 2026-08-09. (Verbatim; that is the entire entry.)

Q5 — Version field semantics — versioned by media type in the Accept header, not by a field

434

"The registry responds with a JSON-formatted string containing metadata for the package named, either in full or abbreviated form depending on what you request in the Accept header. If you provide no Accept header, the full document is returned. To request an abbreviated document with only the fields required to support installation, set the Accept header in your request to the following string: application/vnd.npm.install-v1+json"

435— docs/responses/package-metadata.md, accessed 2026-08-09.

436with a documented graceful-degradation idiom built from HTTP q-values:

437

"A more typical accept header might request json as a fallback, like this: application/vnd.npm.install-v1+json; q=1.0, application/json; q=0.8, */*"

438— same source.

439There is no version field inside the document. NOT FOUND: any schemaVersion, api-version or equivalent in the packument. Searched both registry docs files. NOT FOUND: any documented reader contract for encountering a higher version — there is no install-v2, and no stated policy for what a client does if one appeared.

440Two structural consequences for an at-rest index: (a) this mechanism does not exist without a request — a file in a git repo has no Accept header, so npm's entire versioning strategy is unavailable to a file-at-rest design; (b) it has been an active interop hazard even where it does apply — see §5.

Q6 — Field identity and reuse

441Fields are identified by name. NOT FOUND: any reserved-prefix convention, or any stated policy on renaming/removing/reusing field names. Searched both registry docs and the package.json docs page for reserved, rename, deprecat.

442Observed convention only: registry-generated fields carry a leading underscore (_id, _rev, _npmUser, _npmVersion, _nodeVersion, _hasShrinkwrap) — and two of them are undisguised CouchDB implementation leakage frozen into the public format:

443

"- _id: the package name, used as an ID in CouchDB - _rev: the revision number of this version of the document in CouchDB"

444— docs/responses/package-metadata.md, accessed 2026-08-09. The storage engine's identifiers became permanent public API.

Q7 — Round-trip preservation

445NOT FOUND: any statement about preserving unknown fields on rewrite. Searched both npm registry docs files. The nearest documented guarantee is about trustworthiness of a generated subtree, not preservation:

446

"The dist object is generated by npm and may be relied upon."

447— docs/responses/package-metadata.md, accessed 2026-08-09.

Q8 — Migration mechanism

448Never-break-only-add, plus dual serving via content negotiation, plus per-field "since <date>" annotations (Q3). The only removal documented is a deprecation with a warning period rather than a format break (the licenses array, §5). NOT FOUND: any explicit migration path or format-version transition document for the packument.

Q1 — Tagged vs untagged unions

451type is an untagged, derived discriminator: an open registry key that silently expands into (classifier, extension, packaging).

452[FH] POM XSD maven-4.0.0.xsd, on type:

453

"The type of dependency, that will be mapped to a file extension, an optional classifier, and a few other attributes. Some examples are jar, war, ejb-client and test-jar: see default artifact handlers for a list. New types can be defined by extensions, so this is not a complete list."

454[FH] and on packaging:

455

"Plugins can create their own packaging, and therefore their own packaging types, so this list does not contain all possible types."

456[SS] Maven's own prose names the consequence (https://maven.apache.org/repositories/artifacts.html):

457

"This has "interesting" consequences. Consider the artifact org.project:reusable-test-support:1.0:tests:jar. Maybe surprisingly, a dependency on this artifact can be described in two ways:"

458

"If the type is not one of these 11 names, then the value of the "type" is used as the extension."

459Recorded trouble, still open: [SS] MNG-8097 (created 2024-04-15, Open), "Validate that each dependency->type is a type registered in an artifact handler":

460

"Currently often the dependency's type is being set to the extension and the resolution is lenient, i.e. if there is no artifact handler defining the value given in dependency->type resolution transparently uses the type as extension."

461[SS] MNG-6681 (Fixed 3.6.2) — the documentation of the discriminator was itself wrong:

462

"Reading that, people understand that type is extension" … "The "While it usually represents the extension on the filename of the dependency, that is not always the case." as first explanation is totally misleading"

463[SS] MNG-4654 (Fixed 3.0-alpha-3) — packaging/type conflation shipped wrong file extensions:

464

"Maven ignores the artifact handler because it tries to match project.packaging with artifactHandler.packaging. As a result, the project's main artifacts ends up being installed/deployed with the file extension "woframework" instead of "jar"."

465[SS] And two normative Maven pages disagree on the same mapping (artifact-handlers.html gives ejb-client classifier client; artifacts.html differs) — unreconciled.

466At rest, maven-metadata.xml stores only the expanded pair and never the tag — [FH] repository-metadata-1.1.0.xsd: "The classifier of the sub-artifact. Each classifier and extension pair may only appear once."

Q2 — Absent vs empty vs null

467Empty collections omitted; scalars equal to their default omitted; absent == empty == default. [FH] from Maven's own code generator templates (src/mdo/writer-stax.vm):

468

if (list != null && !list.isEmpty()) · if (props != null && !props.isEmpty()) · if (value != null && !Objects.equals(defaultValue, value))

469One three-state case is modelled, using the empty string as the "unknown" token — [FH] DependencyScope.java:

470

UNDEFINED("", false) — "When no scope is explicitly given, UNDEFINED will be used, but its meaning will depend on whether the DependencyCoordinates is used in dependency management…"

471[FH] Real Central data contradicts the omit rule — a present-but-empty <release></release> appears in ASF snapshot metadata. [SS] And validateEnum short-circuits on "" (if (string == null || string.length() <= 0) { return true; }), so the empty string is silently accepted everywhere.

472Distinct from every JSON subject here: absence in a POM means "inherit from the parent document", not "apply a default" — resolution is by document composition ([FH] https://maven.apache.org/ref/current/maven-model/maven.html: "Values from the parent project will be the default for this project if they are left unspecified.").

Q3 — Closed vocabularies

473[FH] DefaultModelValidator.validateEnum emits:

474

"must be one of " + values + " but is '" + string + "'."

475But the severity was deliberately lowered because third parties invented values — [SS] Maven 3.9.x source comment:

476

java /* * TODO Extensions like Flex Mojos use custom scopes like "merged", "internal", "external", etc. In * order to don't break backward-compat with those, only warn but don't error out. */ validateEnum(prefix, "scope", problems, Severity.WARNING, ...

477[SS] A rendered instance (MNG-5072):

478

[WARNING] 'dependencies.dependency.scope' for junit:junit:jar must be one of [provided, compile, runtime, test, system] but is 'test,runtime'. @ line 13, column 11

479[SS] …while the plugin-dependency variant of the same field is ERROR-severity with a different list (TUSCANY-3845). The same concept, two vocabularies, two severities.

480[FH] New enum members are hard-gated on modelVersion:

481

java // MNG-8750: New dependency scopes are only supported starting with modelVersion 4.1.0 "scope '" + scope + "' is not supported with modelVersion 4.0.0; " + "use modelVersion 4.1.0 or remove this scope.",

482[SS] (Caveat carried forward: the MNG-8750 reference in that comment does not match JIRA — MNG-8750 is an unrelated issue. Correct id NOT FOUND.)

483[FH] The scope vocabulary grew 5 → 10 in Maven 4. [FH] And one vocabulary shrank — combine.children override → merge, combine.self append → merge — requiring a shipped rewrite tool.

484[FH] NOT FOUND: any scope enumeration in either published XSD — scope is xs:string in both 4.0.0 and 4.1.0. The vocabulary lives only in a validator.

Q4 — Strictness — normatively REJECT, but scoped by provenance, with a lenient retry

485[FH] The throw site, Maven's reader template src/mdo/reader-stax.vm:

486

java private void checkUnknownElement(XMLStreamReader parser, boolean strict) throws XMLStreamException { if (strict) { throw new XMLStreamException("Unrecognised tag: '" + parser.getName() + "'", parser.getLocation(), null); } for (int unrecognizedTagCount = 1; unrecognizedTagCount > 0;) { int eventType = nextTag(parser); if (eventType == XMLStreamReader.START_ELEMENT) { unrecognizedTagCount++; } else if (eventType == XMLStreamReader.END_ELEMENT) { unrecognizedTagCount--; } } }

487Strictness also covers attributes and namespace — [FH]:

488

throw new XMLStreamException("Unknown attribute '" + attribute + "' for tag '" + tagName + "'", …) String.format("Unexpected namespace for element '%s': found '%s' but expected '%s'", …)

489And the user-facing framing — [FH] Apache Maven wiki, ModelParseException (https://cwiki.apache.org/confluence/display/MAVEN/ModelParseException):

490

"This error indicates a bad syntax of your POM." "The POM contains invalid/unknown elements that Maven does not understand." "The POM is not even well-formed XML, e.g. the XML elements are not properly nested."

491Unknown element and malformed XML are the same error class — Maven treats unrecognised content as corruption.

492But strictness is scoped by PROVENANCE, and this is the crucial detail — [FH] DefaultModelBuilder: boolean strict = isBuildRequest();. [SS] In Maven 3.9.x it is literally file-vs-repository: a local pomFile → Severity.ERROR; a downloaded POM (pomFile == null) → Severity.WARNING and the build continues. [FH] Maven states the consequence in its own code:

493

// Non-POM projects are consumed as dependencies where unknown elements are ignored…

494This is the same owner-vs-consumer axis Go chose (§1.4 Q4) — strict on the document you author, lenient on documents you merely consume. Maven depends on that leniency to publish 4.1.0 POMs into a 4.0.0 world (Q8).

495[SS] Recorded rejections in the wild: MNG-7837 (Unrecognised tag: 'project' for a legal element inside plugin <configuration>); MNG-6036 (open 2016→2023, namespaced config rejected — reporter: "It would be awesome if the XML parser would resolve namespaces properly."); MNG-8666 (2025) — Maven's own shipped settings.xml unreadable by its own builder: Unrecognised tag: 'repositories'.

496[FH] Schema-level closure: xs:any appears exactly 12× in maven-4.0.0.xsd, only inside properties/configuration/reports/goals, always processContents="skip". No xs:any at Model level; no anyAttribute anywhere.

497[FH] maven-metadata.xml is always read NON-strict: return new MetadataStaxReader().read(input, false); [SS] MNG-4498: "We should just warn in case of unreadable metadata." — the published index is lenient even though the authored POM is not.

498NOT FOUND (both streams): any maven.apache.org page stating normatively that readers reject unknown elements. The norm exists only in generator code plus closed xs:all content models.

Q5 — Version field semantics

499[FH] 4.0.0 XSD: modelVersion required — "4.0.0 is the only value supported by Maven 3." [FH] 4.1.0 XSD: optional (minOccurs="0"), namespace http://maven.apache.org/POM/4.1.0. [FH] master .mdo: required again, "4.0.0 for Maven 3, 4.1.0 or 4.2.0 for Maven 4" — the required/optional discrepancy between the published 4.1.0 XSD and current master is real and unreconciled. [FH] KNOWN_MODEL_VERSIONS = List.of(4.0.0, 4.1.0, 4.2.0); maven-4.2.0.xsd → HTTP 404.

500Reader contract on a higher version — [FH] master:

501

requestedModel + "' is not supported by this Maven version (" + getMavenVersionString(session) + "). Supported modelVersions are: " + validVersions + ". Building this project requires a newer version of Maven."

502[SS] Maven 3.9.x, at Severity.FATAL:

503

"of '" + string + "' is newer than the versions supported by this version of Maven: " + values + ". Building this project requires a newer version of Maven."

504[SS] A real rendered instance (MNG-7851, input 4.0):

505

'modelVersion' of '4.0' is newer than the versions supported by this version of Maven: [4.0.0]. Building this project requires a newer version of Maven.

506[SS] (Caveat: no verbatim log line for 4.1.0 under Maven 3 was found. In practice Maven 3 hits Unrecognised tag: 'subprojects' at parse time before the validator runs — i.e. the strict reader defeats the version field's own error message.)

507modelVersion is a computed minimum, not a declaration — [FH]:

508

String minVersion = new MavenModelVersion().getModelVersion(model); … "the model contains elements that require a model version of " + minVersion,

509and it silently downgrades an over-declaration: [FH] MavenModelVersionTest.testV4ModelVersion — a model declaring 4.1.0 that uses no 4.1.0 feature returns "4.0.0".

510maven-metadata.xml: a version attribute exists, is optional since 1.1.0, and is dead in practice. [FH] XSD: <xs:attribute name="modelVersion" type="xs:string" use="optional"/> — "The version of the underlying metadata model." [FH] Real Central files carry neither namespace nor attribute. [FH] My own live fetch of commons-lang3/maven-metadata.xml (2026-08-09) confirms it: the only document-level scalar is <lastUpdated>20251116125534</lastUpdated>.

511[SS] MNG-4139 (Fixed 2.2.0) — the schema reference was deliberately deleted:

512

"the schema is not used since we don't use a validating parser, and nobody really edits them by hand. With network traffic on these files at a premium for downloads, it's best left out to save some bytes."

513[SS] MNG-3125 — the XSD arrived in 2007, after the format was already in production, with an empty description.

Q6 — Field identity and reuse

514[FH] Identity = element name + namespace, and the namespace is templated from the version: xml.namespace="http://maven.apache.org/POM/${version}". Every model-version bump is a namespace break.

515[FH] Rename policy = add + deprecate + keep. XSD element-name diff 4.0.0 → 4.1.0: only-in-4.1.0 = condition, lang, source, sources, stringFiltering, subproject, subprojects, targetVersion; only-in-4.0.0 = none. modules carries <xs:documentation source="version">[4.0.0 => 4.1.0]</xs:documentation> and @deprecated Use {@link #subprojects} instead., with @Deprecated(since = "4.0.0") and an mdo version range ending at 4.2.0 — i.e. scheduled for removal two model versions later.

516NOT FOUND (both streams): any reserved-name list or written policy on reuse of a retired element name.

Q7 — Round-trip preservation — none, anywhere

517[FH] The non-strict skip loop (Q4) records nothing — unknown subtrees are consumed and discarded, not retained. Writers emit only model fields.

518[FH] MavenMetadata.merge reads recessive → merges → writes from the typed model; a modelVersion attribute on an existing remote file is dropped on the next deploy.

519[FH] The consumer POM is a deliberate lossy projection: "Removes build, mailing lists, issue management, and other build-specific information"; builder.properties(null).reporting(null);

520[FH] The only preservation Maven achieves is of source text, and it needed a third-party library — mvnup "uses domtrip internally for superior formatting preservation".

Q8 — Migration mechanism — dual artifacts + automatic downgrade + a migration CLI

521Not "never break, only add", and not in-place upgrade. [SS] Maven's own site:

522

"Maven 4 updates the POM version to 4.1.0 which defines the namespace http://maven.apache.org/POM/4.1.0. Version 4.1.0 adds some new elements and attributes, while others are marked as deprecated. To not break the ecosystem, this version is only available for the build POM (stored in source control), while the consumer POM (available in remote repositories like Maven Central) will still use version 4.0.0. Maven generates the consumer POM during the build from the build POM."

523

"Warning: There are rare situations where Maven 4 will produce a consumer POM based on version 4.1.0, for example when condition-based profiles (see below) can't be transformed to version 4.0.0. Maven will show a warning in such situations."

524[SS] The originating issue, MNG-6656 (Fixed, 2019-05-14), states the motive plainly:

525

"The pom.xml as we know it has reached it limits, but it is quite hard to do improvements as long as the local pom (as part of the sources) is exactly the same as the file being published. For the Maven eco system it is important that the published file will still be a model 4.0.0 to ensure other projects can still depend on these artifacts."

526[FH] Plus a hard failure when the projection is impossible ("cannot be downgraded to model version 4.0.0 because it contains features that require a newer model version…"), and a shipped one-way upgrade tool: mvnup, "If not specified the tool will target model version 4.0.0." / "Model version 4.0.0 is fully compatible with Maven 3, while projects with a model version 4.1.0 can only be built using Maven 4." — // Support upgrades: 4.0.0 → 4.1.0, 4.0.0 → 4.2.0, 4.1.0 → 4.2.0.

527The generalisable pattern: separate the AUTHORED document from the PUBLISHED document, and let the published one stay on the old schema forever. That is the same move OCI made under duress (§1.2 Q8) and the same one NuGet named as policy (§1.8b Q8).

Q1 — Tagged vs untagged unions — mandatory free-form tag as the record's primary key

531[FH] Every entry carries an explicit type, and the bodies are structurally identical:

532

xml <repomd xmlns="http://linux.duke.edu/metadata/repo" xmlns:rpm="http://linux.duke.edu/metadata/rpm"> <revision>1786236202</revision> <data type="primary"> <checksum type="sha256">14e40c5facb527535365188388722571d48d9fb479ef4ef2fa20aaeaa648c5c5</checksum> <open-checksum type="sha256">26576acd8ca65bdf43a22662e232af24f599b41ee4981ba1c7da3f188ce510d5</open-checksum> <location href="repodata/14e40c5facb527535365188388722571d48d9fb479ef4ef2fa20aaeaa648c5c5-primary.xml.xz"/> <timestamp>1786236183</timestamp> <size>4514744</size> <open-size>55142100</open-size> </data>

533— live EPEL 9 file. The eight types present: primary, filelists, other, primary_db, filelists_db, other_db, group, updateinfo.

534[SS] The tag is the identity — // Remove all existing record of the same type in cr_repomd_set_record.

535But createrepo_c layers two inferred discriminators on top, and both are fragile:

536[SS] (a) payload shape from a string suffix: if (g_str_has_suffix((char *) rec->type, "_db")) { … "database_version" … }

537[SS] (b) the tag derived from the filename when unspecified — modifyrepo_c(8): "Specific datatype of the metadata, will be derived from the filename if not specified." — the code splits the basename at the first ., with no vocabulary check anywhere, only g_warning("Record with type \"%s\" already exists in repomd.xml", task->type);

538[SS] And a missing tag is only a warning, with the reader inventing a value: val = "unknown"; — shipped with a regression fixture <data> <!-- WARNING: Missing data element type -->. Collision hazard: two untagged records both key to "unknown" and evict each other.

Q2 — Absent vs empty vs null

539[SS] Omitted when absent, with NULL / -1 sentinels meaning "not recorded": if (rec->checksum_open) · if (rec->size_open != -1) · if (rec->checksum_header && rec->size_header != -1). There is no empty form and no way to say "empty but known".

540[SS] <revision> is the one synthesised field: // Use the current time if no revision was explicitly specified.

541NOT FOUND: any document stating an absent-vs-empty rule. The rule exists only as C code.

Q3 — Closed vocabularies — <data type> is OPEN; the checksum vocabulary is where it went wrong

542[FH] The canonical client-side statement, dnf5.conf(5) on optional_metadata_types:

543

"Note: The list includes only metadata types recognized by DNF. However, a repository's metadata may include various other types (e.g., AppStream or metadata stored as databases instead of XML files). The special value all represents all metadata types present in the repository, including those unknown to DNF."

544[FH✓] Zchunk is the textbook additive rollout — Fedora Change, Upgrade/compatibility impact:

545

"Old-style xz and gzip metadata will continue to be generated by createrepo_c, so old versions of Fedora will use the xz/gzip metadata while upgrading to the new version, and then use the new zchunk metadata after the upgrade."

546[SS] Note what the proposal does not say: anything about what a reader that does not understand *_zck does. The entire compatibility story is "we keep emitting the old records too." New types are minted by string concatenation, even for third-party records: gchar *additional_metadatum_rec_zck_type = g_strconcat(elem_type, "_zck", NULL);

547The checksum trap — the most important RPM finding. [FH] createrepo(8) (https://linux.die.net/man/8/createrepo, header .TH "createrepo" "8" "2005 Jan 2" "Seth Vidal"):

548

"-s --checksum Choose the checksum type used in repomd.xml and for packages in the metadata. The default is now "sha256" (if python has hashlib). The older default was "sha", which is actually "sha1", however explicitly using "sha1" doesn't work on older (3.0.x) versions of yum, you need to specify "sha"."

549and the workarounds, same file [FH]:

550

"--unique-md-filenames Include the file's checksum in the metadata filename, helps HTTP caching (default)" "--simple-md-filenames Do not include the file's checksum in the metadata filename." "--retain-old-md Keep around the latest (by timestamp) N copies of the old repodata (so clients with older repomd.xml files can still access it). Default is 0."

551Two stacked breakages: the default moved to an algorithm old clients could not compute, and the two spellings of the same algorithm are not interchangeable — old yum accepts sha, rejects sha1. [SS] The alias survives in code to this day (else if (!strcmp(sha_type, "")) return CR_CHECKSUM_SHA; → EVP_sha1()), with the header admitting "Note: SHA is just a "nickname" for the SHA1. This is for the compatibility with original createrepo.", and is mirrored in librepo and libsolv. [SS] When it landed: createrepo ChangeLog, 2009-01-26 Seth Vidal — "make sha256 the default checksum type everywhere".

Q4 — Strictness — readers IGNORE, uniformly across three generations

552[SS] createrepo_c: // We are inside of unknown element → descend-and-discard; cr_xml_parser_warning(pd, CR_XML_WARNING_UNKNOWNTAG, "Unknown element \"%s\"", element); → warn and continue.

553[SS] Original yum — the namespace is stripped, not checked, and the dispatch is an if/elif chain with no else:

554

python def ns_cleanup(qn): if qn.find('}') == -1: return qn return qn.split('}')[1]

555with only well-formedness fatal: raise RepoMDError, "Damaged repomd.xml file". Every <data> record is retained keyed by its type string whether understood or not.

556[SS] librepo is the same design; libsolv swallows unknowns silently (its debug print is #if 0'd out) and its element table is narrower than createrepo_c's — so everything zchunk added falls into that path.

557[SS] The one hard failure is an unknown checksum type: libsolv pool_error(…, "line %d: unknown checksum type: %s", …); librepo LRE_UNKNOWNCHECKSUM.

558Asymmetry, and it is the exact opposite of Maven's: the writer is strict ([SS] "Unknown/Unsupported checksum type \"%s\""), the reader is permissive.

559NOT FOUND: any normative "readers MUST ignore" statement — there is no specification document to state it in.

Q5 — Version field semantics — no schema version; the one schema that exists is a stale, unenforced DTD written by a consumer

560[FH] The only document-level scalar in the live file is <revision>1786236202</revision> — a Unix timestamp. [SS] And the man pages say it is arbitrary: "Arbitrary string for a repository revision." / "User-specified revision for this repository." Observed values differ in kind across distros: <revision>9-stream</revision> (CentOS) vs <revision>1786236202</revision> (EPEL 9) — [SS]/[FH].

561[FH] The root element carries two fixed, unversioned namespaces: xmlns="http://linux.duke.edu/metadata/repo".

562[FH✓] A schema does exist, and its provenance is the finding. yum/docs/repomd.dtd:

563

"This is DTD for repomd.xml files used by yum. Author: Miroslav Suchý <msuchy@redhat.com> Date: 2010-01-29"

564

<!ATTLIST data type (other_db | other | filelists_db | filelists | primary_db | primary | group | prestodelta | group_gz) #REQUIRED

565Written by a consumer, five years after the format was already in production, declaring type as a closed enumeration of nine values. Reality blew past it immediately (updateinfo, modules, every *_zck, filelists-ext, …), and it lacks <header-checksum>, <header-size>, <repoid>, <contenthash>, <tags> entirely. No implementation validates against it. [SS] Its origin, the author on the yum list 2009-06-29:

566

"BTW - Do you know where is definition of repodata files (repomd.xml, primary.xml...)? I could not find DTD file, nor any other documentation of the format."

567[SS] The only version field anywhere is <database_version>, per-record and payload-only, emitted on a _db suffix test. No major/minor split anywhere.

Q6 — Field identity and reuse

568[SS] Identity = the type string; namespaces are declared but unenforced. A rename happened and the old name is a permanent alias: gchar *repoid; /*!< OBSOLETE, replaced by contenthash */ — both still read and written. Nothing is ever removed.

569[FH] The namespace URI outlived its host: http://linux.duke.edu/metadata/repo is still on every RPM repo on the internet — unversioned, therefore unchangeable.

570NOT FOUND: reserved-name list or reuse policy. Collision avoidance is purely social — with a failure mode documented in Q7.

Q7 — Round-trip preservation — preserved by default, with two silent-loss paths

571[SS] The default: "Keep all additional metadata (not primary, filelists and other xml or sqlite files, nor their compressed variants) from source repository during update (default)."; .keep_all_metadata = TRUE,

572[SS] Loss path 1 — it is gated on --update:

573

c if (cmd_options->update && cmd_options->keep_all_metadata && old_metadata_location && old_metadata_location->additional_metadata)

574A plain createrepo_c <dir> regenerates from scratch and third-party records are gone.

575[SS] Loss path 2 — "additional" is a hard-coded prefix test, not a registry:

576

c else if ( !g_str_has_prefix(record->type, "primary_" ) && !g_str_has_prefix(record->type, "filelists_" ) && !g_str_has_prefix(record->type, "filelists-ext_" ) && !g_str_has_prefix(record->type, "other_" ) )

577A third-party record whose type merely starts with one of those four prefixes is silently dropped on every update, with no warning. Four undocumented reserved prefixes, discoverable only by reading C.

578Either way, preservation is at record granularity — unknown child elements inside a known <data> are dropped.

579Separately, the format achieves verification-without-comprehension by content-addressing every payload filename — [FH] createrepo(8): "--unique-md-filenames Include the file's checksum in the metadata filename", visible in the live file as repodata/14e40c5f…c5c5-primary.xml.xz. A reader that understood 3 of 8 entries can still verify exactly the 3 it fetched.

Q8 — Migration mechanism — "never break, only add", because there is no version to negotiate

580[FH✓] Every change has been a new type alongside the old (zchunk), or a default flip with restore flags (--checksum, --simple-md-filenames, --retain-old-md). [SS] Deprecation is a slow fade, never removal — sqlite *_db is documented DEPRECATED and still shipping in EPEL 9 today, as my own live fetch confirms.

581[SS] And the reason the discipline holds is itself the most interesting finding — yum list, 2009-06, on a proposal to add a second checksum per <data>. Seth Vidal:

582

"if we start doing this we run the risk of breaking any of the non-yum depsolvers."

583James Antill:

584

"I think it's worse than that, for instance does anyone know what 3.0.1 will do? Or 2.4.0?"

585With no version field, nobody could answer what deployed readers would do — so the extension was not made. The absence of a version field did not merely fail to help; it exerted a chilling effect on evolution. That is the cost of "no version" stated by the people paying it.

Q1 — Tagged vs untagged unions

589NOT FOUND: a discriminated-union construct in the Simple API JSON. The format is a fixed set of typed keys. The serialization itself is tagged, however — by media type, and the reference client dispatches on it and errors on anything unrecognised:

590

python match content_type: case "application/vnd.pypi.simple.v1+json": handle_v1_json(resp) case "application/vnd.pypi.simple.v1+html" | "text/html": handle_v1_html(resp) case _: raise Exception(f"Unknown content type: {content_type}")

591— PEP 691, accessed 2026-08-09.

Q2 — Absent vs empty vs null

592PEP 691 states shape constraints normatively, and notably forbids the top-level array form:

593

" All JSON responses will always* be a JSON object rather than an array or other type."

594

"* While JSON doesn't natively support an URL type, any value that represents an URL in this API may be either absolute or relative as long as they point to the correct location. If relative, they are relative to the current URL as if it were HTML."

595— PEP 691, accessed 2026-08-09.

596NOT FOUND: an explicit rule on omitting vs emitting []/{}, or on distinguishing empty from unknown. Searched pep-0691.rst for empty, omit, null, absent.

597The "always an object, never an array" rule is worth lifting on its own: a top-level array has nowhere to put meta, so it forecloses ever versioning the document. PEP 691 closed that door deliberately.

Q3 — Closed vocabularies

598NOT FOUND in PEP 691 as a general rule. The relevant live case is hashes, which are carried as a map keyed by algorithm name (hashes: {"sha256": "..."}) rather than a single tagged string — an open key space by construction, consistent with C3 in §3.

Q4 — Strictness — normative, unambiguous, and stated as a MUST on readers

599

" Additional keys may be added to any dictionary objects in the API responses and clients MUST* ignore keys that they don't understand."

600— PEP 691, accessed 2026-08-09.

601This is the clearest single sentence found in the entire research on unknown-field handling for at-rest data. Compare OCI, which says the same thing but downgraded it to SHOULD NOT in v1.1 (§1.2 Q4), and every other subject, which does not say it at all.

602Note the symmetry: the same sentence grants writers permission ("may be added") and binds readers ("MUST ignore"). One sentence, both halves of the contract.

Q5 — Version field semantics — meta.api-version, and the only complete reader contract found

603PEP 691 mandates the field and delegates its semantics:

604

"* All JSON responses will have a meta key, which contains information related to the response itself, rather than the content of the response."

605

"* All JSON responses will have a meta.api-version key, which will be a string that contains the :pep:629 Major.Minor version number, with the same fail/warn semantics as defined in :pep:629."

606— PEP 691, accessed 2026-08-09.

607PEP 629 supplies the semantics — quoted here in full because it is the deliverable of this research:

608

"When interpreting the repository version: Incrementing the major version is used to signal a backwards incompatible change such that existing clients would no longer be expected to be able to meaningfully use the API. Incrementing the minor version is used to signal a backwards compatible change such that existing clients would still be expected to be able to meaningfully use the API."

609and the client obligations:

610

"Clients interacting with the simple API SHOULD introspect each response for the repository version, and if that data does not exist MUST assume that it is version 1.0. When encountering a major version greater than expected, clients MUST hard fail with an appropriate error message for the user. When encountering a minor version greater than expected, clients SHOULD warn users with an appropriate message. Clients MAY still continue to use feature detection in order to determine what features a repository uses."

611— PEP 629, accessed 2026-08-09.

612Three things no other subject does:

1. 613Absent version has a defined meaning — MUST assume 1.0 — so unversioned legacy data stays legal forever.
2. Major and minor have different, explicitly different reader actions: hard fail vs warn. Nobody else splits the reader's response by component.
3. Feature detection is explicitly permitted alongside the version, rather than replaced by it.

614And the expectation that the major will never move — the version exists mostly to disambiguate:

615

"It is expectation of this PEP that the major version will never be incremented, and any future major API evolutions would utilize a different mechanism for API evolution. However the major version is included to disambiguate with future versions (e.g. a hypothetical simple api v2 that lived at /v2/, but which would be confusing if the repository-version was set to a version >= 2)."

616— PEP 629, accessed 2026-08-09.

617PEP 691 adds the writer-side rule, and is candid that the boundary is a judgement call:

618

"Similar to :pep:629, the major version number MUST be incremented if any changes to the new format would result in no longer being able to expect existing clients to meaningfully understand the format. Likewise, the minor version MUST be incremented if features are added or removed from the format, but existing clients would be expected to continue to meaningfully understand the format. Changes that would not result in existing clients being unable to meaningfully understand the format and which do not represent features being added or removed may occur without changing the version number. This is intentionally vague, as this PEP believes it is best left up to future PEPs that make any changes to the API to investigate and decide whether or not that change should increment the major or minor version."

619— PEP 691, accessed 2026-08-09.

620Note the third clause: a change that neither adds nor removes a feature does not bump anything at all. That matches crates.io's practice (§3 C2) but here it is written down.

621Multi-serialization rule, relevant if you ever publish the same data two ways:

622

"Future versions of the API may add things that can only be represented in a subset of the available serializations of that version. All serializations version numbers, within a major version, SHOULD be kept in sync, but the specifics of how a feature serializes into each format may differ, including whether or not that feature is present at all."

623

"It is the intent of this PEP that the API should be thought of as URL endpoints that return data, whose interpretation is defined by the version of that data, and then serialized into the target serialization format."

624— PEP 691, accessed 2026-08-09.

Q6 — Field identity and reuse

625Keys are identified by name. NOT FOUND: a reserved-prefix convention or a stated rename policy in PEP 691. PEP 714, however, performed a rename and legislated exactly how readers and writers handle both names during the overlap — see Q8 and §5.

Q7 — Round-trip preservation

626NOT FOUND: any statement requiring a rewriting client or mirror to preserve unknown keys. Searched pep-0691.rst and pep-0629.rst. Note that PEP 629's rejection of header-based versioning (§5) was motivated by keeping mirrors byte-dumb, which achieves preservation as a side effect — a mirror that copies files verbatim cannot drop anything.

Q8 — Migration mechanism

627Three distinct mechanisms, all exercised:

1. 628Minor bump + MUST-ignore for ordinary additions (Q4, Q5).
2. Dual serialization with content negotiation (HTML + JSON) — with the fallbacks made explicit, and with static-file-friendly alternatives (a URL parameter) offered because negotiation is not always available.
3. Rename the key when an old reader hard-fails on it — PEP 714, with a precisely specified overlap period:

629

"The :pep:658 metadata, when used in the :pep:691 JSON representation of the Simple API, MUST be emitted using the key core-metadata, with the supported values remaining the same."

630

"To support clients that used the previous key names, the HTML representation MAY also be emitted using the data-dist-info-metadata, and if it does so it MUST match the value of data-core-metadata."

631

"Clients consuming the JSON representation of the Simple API MUST read the :pep:658 metadata from the key core-metadata if it is present. They MAY optionally use the legacy dist-info-metadata key if it is present but core-metadata is not."

632— PEP 714, accessed 2026-08-09.

633That is a complete, reusable rename protocol: new name is MUST-read and MUST-write; old name is MAY-write and MAY-read; if both are written they MUST agree. And the graceful-degradation claim is stated rather than assumed:

634

"There is a minor compatibility break in this PEP, in that clients that currently correctly handle the existing metadata keys will not automatically understand the newer metadata keys, but they should degrade gracefully, and simply act as if the :pep:658 metadata does not exist."

635— PEP 714, accessed 2026-08-09.

Q1 / Q3 — @type as discriminator, and a closed vocabulary that failed silently for six years

637NuGet's index is a list of resources each tagged with @type, matched by exact string, with the version baked into the tag value (PackageBaseAddress/3.0.0, SearchQueryService/3.5.0). The designer's own diagnosis of why that was a mistake — NuGet/Home#4206, "Support for v3 versioned services", emgarten, opened 2017-01-05 (metadata verified independently):

638

"The previous model used hardcoded type strings which made it impossible for a server to optimize or support or new feature without breaking older clients unless the client added a new type."

639

"The above scenarios work today in 3.5.0 but cannot be enabled without breaking older clients. Having this feature will make it possible to do the server work at a later date instead of when client work begins on a new version. It would also make it possible for 3rd party servers to support new features instead of being limited to what nuget.org supports."

640Versioning inside the discriminator value couples server capability to client release cadence — the opposite of what it looks like it does. The consequence, live in 2026: SearchFilter.PackageTypes

641

"has been silently ignored by nuget.org and other compliant V3 sources for as long as the property has existed publicly"

642— NuGet.Client PR #7395, merged 2026-06-10 (single-sourced; corroborated by the shipped code and Strings.resx read first-hand by the delegated agent). The fix converts a silent wrong answer into a thrown exception — i.e. the repair for a silent compatibility failure is itself a breaking change.

643The open design question sat unanswered for six years — NuGet/Home#8915, zivkan, 2020-07-20:

644

"From an API design point of view, how should NuGet.Protocol behave when the server implements an old version of the NuGet HTTP protocol and doesn't support packageType filtering?" … "Silently ignore the issue, make the request anyway, and return results that may not be the package type requested?" … "API design is hard :("

Q3 (cont.) / Q6 — a two-tier vocabulary: documented keys are permanent, undocumented keys are disposable

645nuget.org's published policy, per the delegated report:

646

"If a resource is not documented here, we strongly recommend that you do not take a dependency on them. We may remove or change the behavior of these undocumented resources"

647and removals have in fact happened — MetricsService, TotalStats/3.0.0-rc and the bare SearchGalleryQueryService appear in 2015/2016 Wayback captures and are absent from the live index (verified independently by the agent against web.archive.org/…id_/ raw captures and a live fetch, 2026-08-09).

648Saying in writing which keys are contractual and which are yours to churn is cheap, and it is the only mechanism found that lets a published vocabulary shrink.

Q5 — the version field moved twice, and once backwards

649Verified sequence: 3.0.0-rc.1 (Feb 2015) → 3.0.0-beta.1 (Jan 2016) → 3.0.0 (by Aug 2018). Under SemVer 2.0.0 precedence 3.0.0-beta.1 < 3.0.0-rc.1, so the declared schema version went down, and nothing noticed — the client only ever compares version.Major != 3.

650A version field that no reader fully evaluates is not a contract, it is decoration.

Q8 — breaking change ⇒ new URL, never a new major in the same document

651NuGetGallery#5403, "NuGet V3 API service index moving to 3.0.0!", joelverhagen, 2018-02-05 (metadata verified independently):

652

"Non-breaking changes to the schema can be introduced and will be signaled by a minor version change in the existing document" … "Any future breaking changes will not be made to the existing V3 service index. They would be introduced in a different service index URL."

653Directly comparable to Debian's new-file-extension trick (§1.3 Q8) and PEP 629's parenthetical about "a hypothetical simple api v2 that lived at /v2/". For a git-hosted index a new path is nearly free, which makes this the cheapest available answer to "what if we need a real break".

Q7 — the reader normalises rather than preserves

654Per the delegated report on NuGet's System.Text.Json converter test table: non-string array members are silently dropped, a wrong-typed @type degrades to [] with no error, and on write a single-element array is re-emitted as a bare string. A NuGet-shaped round-trip is lossy and shape-changing by design.

Q2, Q4 — NOT FOUND (not recovered from the delegated research)

655The base sweep covering absent-vs-empty and the normative strictness rule was not delivered. Observed evidence only: Azure Artifacts ships four vendor-private @type values (VssBaseUrl, VssFeedId, VssQualifiedFeedViewId, AzureDevOpsProjectId) that no documented client understands, which means unknown entries are tolerated in practice.

NuGet (partial — delta only; see §1.8b)

Homebrew

664Q1–Q8: NOT FOUND across the board. No documented versioning, stability, deprecation, or unknown-field policy exists in the published API docs. A genuine negative result.