Document versioning policy

[ddbeck]

As of v1.0 we're now obliged to make SemVer MAJOR releases for "breaking" changes. We should clearly document what events constitute a breaking change, both to help consumers of the package understand what is needed each time they upgrade and to help maintainers prepare useful release notes.

To that end, we have at least two "obvious" paths:

1. Breaking changes are backwards-incompatible changes to the JSON Schema (e.g., renaming an existing key). This is roughly how @mdn/browser-compat-data does versioning, with some documented exceptions.
2. Breaking changes are backwards-incompatible changes to the JSON Schema (as above) and changes to feature IDs (e.g., renaming a feature, such as css-grid to grid). This is roughly how webref and browser-specs handle it.

This also has implications for minor and patch releases (e.g., under route 1, new features constitute a SemVer PATCH release, but under route 2 those are MINOR releases).

[dontcallmedom]

I think (2) would help deal with #91 and related discussions around evolving features; it allows data consumers to rely on a clear signal well integrated in dependency management tool to determine how much a particular release needs their attention.

[captainbrosset]

I agree that (2) seems better in our case.

I would also note that we should think of an additional communication method for consumers who do not use our package. WPT, for example, tags tests with web-features IDs, but does not, as far as I know, consume the package. How would WPT (and others like it) know when an ID changes?

[ddbeck]

I can't tell how WPT is getting the web-features JSON (I found some code where the process it, but not where the actual dependency to web-features lives). I'd be interested to know what's going on there in more detail.

I imagine that fixing #1186 would at least put non-npm consumers on an equal footing—they'd be able to see that we made a breaking change via version number.

I'd like to get more consumer input before going down this path, but another possibility would be to bake the feature IDs into the schema files. That is, every new, renamed, or removed feature would result in a change to the JSON Schema, while feature churn (e.g., new browser releases) wouldn't.

Longer term, resolving #91 within the schema and with some utilities would be the best thing (e.g., tombstone removed/moved features for a time, then do periodic breaking changes on some published schedule).

[ddbeck]

In the meantime, I've talked myself into documenting and doing option (2). It's only slightly more work than just assuming those keys are volatile and we already have some tooling to help in this area already in the scripts directory.

[ddbeck]

I've opened a draft PR to cover this for the maintainers' publishing process #1725 but it's currently blocked by #1650.

[jcscottiii]

I can provide some insights on WPT's usage of web-features:

The WPT project does not use the NPM package or the data.json. It is up to developers to make sure the feature ID matches one from the repository. As you can imagine, this can lead to out of sync information that has to be manually cleaned up. One future idea is to vendor the data.json and have the lint rules check that ID is still valid.

[jcscottiii]

For data.json consumers like webstatus.dev (non-npm):

The primary concern is ensuring compatibility with the data structure. A clear versioning scheme that signals potential breaking changes to the schema is essential.

Path 1 (Schema-based breaking changes) would be sufficient for us. If a new version indicates a schema change, our parser can avoid attempting to read the data, preventing potential errors.

Path 2 (Schema + Feature ID breaking changes) could also work, but we'd need to handle version ranges. If major versions can include non-schema changes, we'd need to check if the current version falls within a compatible range to ensure proper parsing.

Key takeaway: The most important factor is clear communication about breaking changes through versioning, allowing consumers to adapt their parsing logic accordingly.

[ddbeck]

@jcscottiii Thank you! That's very helpful. I can definitely clearly announce (and document) any schema changes, so you can decide if you need to update your parser. But looking more long-term:

Path 2 (Schema + Feature ID breaking changes) could also work, but we'd need to handle version ranges. If major versions can include non-schema changes, we'd need to check if the current version falls within a compatible range to ensure proper parsing.

Hmm, perhaps a way to signal breaking schema independent of the overall package version, might be useful. The JSON Schema docs suggest setting the $id field in our JSON Schema file (data.schema.json, in the GitHub release artifacts) is the way to do this.

Perhaps once we have a static site for web-features set up, we could publish the schemas there as well.