Make it possible to have a deterministic order in serialized OpenApi files

[gturri]

Is your feature request related to a problem? Please describe.
I use OpenAPI.NET to serialize instances of OpenApiDocument and store them in a git repo.
So, ideally I would like that when I make a simple change to my API, I get a simple change in the serialized document, in order to have an easy to review git diff.

It is not the case currently, because serialization iterates on some Dictionary and that the order of the iteration is hence not deterministic.
For instance, if I add one endpoint to my API, I would like that my serialized OAS file looks like what it was at my previous generation, with just a few added lines. But in practice I may have more diffs, because my paths and my schemas may end up serialized in a different order.

Describe the solution you'd like
I can already achieve part of what I would like from my client code in the sense that I can get a deterministic order for the Schemas section, if I do

OpenApiDocument doc = BuildMyDoc();
doc.Components.Schemas = new SortedList<string, OpenApiSchema>(doc.Component.Schemas) // <-- that's the important line
doc.SerializeAsV3(new OpenApiYamlWriter(....));

It works rather fine, in the sense that it leads to a serialized schemas section where all schemas are order alphabetically (making it nice for my git diffs, and making the file easier to browse overall). However, it does not feel really "clean" in itself, and this approach does not work for other parts of the document. In particular for the paths; because they are represented by an OpenApiPaths which ends up extending a Dictionary and I could not find a virtual method or an attribute I could override to achieve a similar deterministic order.

Given this context, a solution that would be nice would be to edit IOpenApiSerializable in order to turn SerializeAsV3(IOpenApiWriter writer) into SerializeAsV3(IOpenApiWriter writer, bool enforceAlphabeticalOrder = false).
So for instance OpenApiExtensibleDictionary.SerializeAsV3 could be implemented like

var items = enforceAlphabeticalOrder ? new SortedList<string, T>(this) : this;
foreach(var item in items) {
  ...
}

Having this feature behind a boolean enforceAlphabeticalOrder could make it possible for users interested in this feature to use it, and users not interested in it would not suffer any performance penalty by leaving it to false.

Describe alternatives you've considered
As explained above, I tried to achieve this order from my client code, but faced some hard limitations

Additional context
If the proposed approach looks good to you, and if it could help, I guess I could try to propose a pull request to implement this.

[gturri]

FWIW (in case someone has the same issue and stumble on this feature request), I found a workaround, which is in a nutshell:

• serialize the OpenApiDocument to a yaml string
• deserialize this string to a Dictionary with YamlDotNet
• sort the values of that Dictionary
• then serialize that Dictionary to get the file I was looking for.

From a code point of view it looks like this:

var oas = ...

// Ensure the schemas are sorted. This can be done directly on the OpenApiDocument
oas.Components.Schemas = new SortedList<string, OpenApiSchema>(oas.Components.Schemas, StringComparer.OrdinalIgnoreCase);

// Now the trick to sort the paths
using var oasWriter = new StringWriter();
oas.SerializeAsV3(new OpenApiYamlWriter(oasWriter));
var yamlDoc = (Dictionary<object, object>) new YamlDotNet.Serialization.Deserializer().Deserialize(new StringReader(oasWriter.ToString()));
yamlDoc["paths"] = ToDictionaryWithSortedKeyStrings((Dictionary<object, object>) yamlDoc["paths"]);

// Now I can serialize back (with YamlDotNet this time)
using var textWriter = new StreamWriter(outPath);
new YamlDotNet.Serialization.Serializer().Serialize(textWriter, yamlDoc);


// A trick here is that we get a IDictionary<object, object> but we know that the keys are string, and we need to
// cast them in order to apply a deterministic order to it (because if I don't use StringComparer.OrdinalIgnoreCase
// I may end up with a different behavior on my laptop and on my CI
private static SortedList<string, object> ToDictionaryWithSortedKeyStrings(IDictionary<object, object> dictionary)
{
    var sorted = new SortedList<string, object>(StringComparer.OrdinalIgnoreCase);
    foreach (var kvp in dictionary)
    {
        sorted.Add(kvp.Key.ToString(), kvp.Value);
    }
    return sorted;
}

That being said, I guess it makes sense to leave this feature request open, because it would be much more convenient if this could be supported directly by OpenAPI.NET

[mikekistler]

Related: dotnet/aspnetcore#59809

[captainsafia]

@RachitMalik12 Can we slot this in for v2?

I don't think we necessarily have to change anything about the in-memory model to support sorting but the serialization step should attempt do lexogaphic sorting on the path values before seralizing the document.

Similarily, we should probably lexographic sort the elements when in each component type? For example, all schemas get sorted alphabetically by ID and all security schemes as well.

[darrelmiller]

@captainsafia I think I would rather the default behaviour to be to maintain the order provided by the user, but we would be open to providing a Sort() method to reorder the dictionary.

[RachitMalik12]

Relevant for all dictionary based collections

[baywet]

My assumption here was that we'd provide either a comparer or a callback on the settings object so it can be used during the serialization.
But @Michael-Wamae 's interpretation was to implement sorting methods here.
@darrelmiller can you clarify please?

[darrelmiller]

@baywet There are two distinct work items here.

1. Change the behaviour of dictionaries/lists to maintain the order that items were added.
2. Add a helper method to allow users to reorder the contents of the dictionaries/lists.

[baywet]

To maintain the order we could replace all the Dictionaries we have by OrderedDictionary I thought it was only available for net5+ but it goes all the way back to netstandard2.0, which is good news for us.

As for the helper, I'm assuming this is only relevant during serialization? there are no other scenarios which would require control over ordering from the user? correct?

We have other places where we use lists, they maintain order. We also have hashsets, they don't have an OrderedSet equivalent. Would you leave it as is? or also require to use an ordering collection instead?

[darrelmiller]

@baywet OrderedDictionary sounds good. Yes I think the ordering is only necessary during serialization, but why does it matter if we just have a helper method? Are you thinking of doing this a different way, like in the writer settings?

I looked at some places where we have HashSet and I think we may need to change this too. We use it for Tags and considering Tags are used for generating documentation, the order might end up being significant to doc rendering. Are there any downsides to changing from HashSet? Is it just that we lose the uniqueness constraint?