Relative reference in subdirectory OpenApi document fails to load

[dldl-cmd]

Relative reference in subdirectory OpenApi document fails to load

When loading a document which is splitted into multiple documents relative references are failing to load when a parent document is in a subdirectory.

What happens is, that a System.IO.FileNotFoundException is thrown:

Could not find file '<path to root of document structure>\Pets.json'.
StackTrace:
   at Microsoft.Win32.SafeHandles.SafeFileHandle.CreateFile(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options)
   at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
   at System.IO.Strategies.OSFileStreamStrategy..ctor(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
   at System.IO.Strategies.FileStreamHelpers.ChooseStrategyCore(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
   at System.IO.File.OpenRead(String path)
   at Microsoft.OpenApi.Readers.Services.DefaultStreamLoader.<LoadAsync>d__4.MoveNext() in C:\Users\user\source\repos\OpenAPI.NET\src\Microsoft.OpenApi.Readers\Services\DefaultStreamLoader.cs:line 47
   at Microsoft.OpenApi.Readers.Services.OpenApiWorkspaceLoader.<LoadAsync>d__4.MoveNext() in C:\Users\user\source\repos\OpenAPI.NET\src\Microsoft.OpenApi.Readers\Services\OpenApiWorkspaceLoader.cs:line 46
   at Microsoft.OpenApi.Readers.OpenApiYamlDocumentReader.<ReadAsync>d__3.MoveNext() in C:\Users\user\source\repos\OpenAPI.NET\src\Microsoft.OpenApi.Readers\OpenApiYamlDocumentReader.cs:line 105
   at Microsoft.OpenApi.Readers.OpenApiTextReaderReader.<ReadAsync>d__3.MoveNext() in C:\Users\user\source\repos\OpenAPI.NET\src\Microsoft.OpenApi.Readers\OpenApiTextReaderReader.cs:line 83
   at Microsoft.OpenApi.Readers.OpenApiStreamReader.<ReadAsync>d__3.MoveNext() in C:\Users\user\source\repos\OpenAPI.NET\src\Microsoft.OpenApi.Readers\OpenApiStreamReader.cs:line 71
   at Microsoft.OpenApi.Readers.Services.OpenApiWorkspaceLoader.<LoadAsync>d__4.MoveNext() in C:\Users\user\source\repos\OpenAPI.NET\src\Microsoft.OpenApi.Readers\Services\OpenApiWorkspaceLoader.cs:line 47
   at Microsoft.OpenApi.Readers.OpenApiYamlDocumentReader.<ReadAsync>d__3.MoveNext() in C:\Users\user\source\repos\OpenAPI.NET\src\Microsoft.OpenApi.Readers\OpenApiYamlDocumentReader.cs:line 105
   at Microsoft.OpenApi.Readers.OpenApiTextReaderReader.<ReadAsync>d__3.MoveNext() in C:\Users\user\source\repos\OpenAPI.NET\src\Microsoft.OpenApi.Readers\OpenApiTextReaderReader.cs:line 83
   at Microsoft.OpenApi.Readers.OpenApiStreamReader.<ReadAsync>d__3.MoveNext() in C:\Users\user\source\repos\OpenAPI.NET\src\Microsoft.OpenApi.Readers\OpenApiStreamReader.cs:line 71
   at Program.<<Main>$>d__0.MoveNext() in C:\Users\user\source\repos\OpenApiReferencesInSubDirectories\OpenApiReferencesInSubDirectories\Program.cs:line 17

Files to reproduce

OpenApi.NET Version: 1.6.14

using Microsoft.OpenApi.Readers;

OpenApiStreamReader openApiStreamReader = new OpenApiStreamReader(new OpenApiReaderSettings
{
    BaseUrl = new Uri("<path to root of document structure>"),
    ReferenceResolution = ReferenceResolutionSetting.ResolveLocalReferences,
    LoadExternalRefs = true,
});

FileStream fileStream = new FileStream("Root.json", FileMode.Open, FileAccess.Read);

var document =  await openApiStreamReader.ReadAsync(fileStream);

./Root.json

{
  "openapi": "3.0.0",
  "info": {
    "version": "1.0.0",
    "title": "Swagger Petstore",
    "license": {
      "name": "MIT"
    }
  },
  "servers": [
    {
      "url": "http://petstore.swagger.io/v1"
    }
  ],
  "paths": {
    "/pets": {
      "get": {
        "summary": "List all pets",
        "operationId": "listPets",
        "responses": {
          "200": {
            "description": "An array of pets",
            "content": {
              "application/json": {
                "schema": {
                  "$ref": "./Directory/AllPets.json#/components/schemas/AllPets"
                }
              }
            }
          }
        }
      }
    }
  }
}

./Directory/AllPets.json

{
  "openapi": "3.0.0",
  "info": {
    "version": "1.0.0",
    "title": "Swagger Petstore"
  },
  "paths": {
  },
  "components": {
    "schemas": {
      "AllPets": {
        "$ref": "./Pets.json#/components/schemas/Pets"
      }
    }
  }
}

./Directory/Pets.json

{
  "openapi": "3.0.0",
  "info": {
    "version": "1.0.0",
    "title": "Swagger Petstore"
  },
  "paths": {
  },
  "components": {
    "schemas": {
      "Pets": {
        "type": "array",
        "items": {
          "$ref": "./Pet.json#/components/schemas/Pet"
        }
      },
      "Pet": {
        "required": [
          "id",
          "name"
        ],
        "properties": {
          "id": {
            "type": "integer",
            "format": "int64"
          },
          "name": {
            "type": "string"
          },
          "tag": {
            "type": "string"
          }
        }
      }
    }
  }
}

[baywet]

closing since #2133 was merged

[dldl-cmd]

@baywet The issue is still present. As the parameter LoadExternalRefs is set to true. If the parameter is set to false, the issue would be solved by the PR #2133 as it only fixed that the reference could be parsed correctly.

With the current main branch, the following exception is thrown:

System.IO.FileNotFoundException
  HResult=0x80070002
  Message=Could not find file 'C:\Users\...\source\repos\ReferencesToSubFolders\ReferencesToSubFolders\Pets.json'.
  Source=System.Private.CoreLib
  StackTrace:
   at Microsoft.Win32.SafeHandles.SafeFileHandle.CreateFile(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options)
   at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
   at System.IO.Strategies.OSFileStreamStrategy..ctor(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
   at System.IO.Strategies.FileStreamHelpers.ChooseStrategyCore(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode)
   at System.IO.File.OpenRead(String path)
   at Microsoft.OpenApi.Reader.Services.DefaultStreamLoader.<LoadAsync>d__3.MoveNext() in ...\OpenAPI.NET\src\Microsoft.OpenApi\Reader\Services\DefaultStreamLoader.cs:line 45
   at Microsoft.OpenApi.Reader.Services.OpenApiWorkspaceLoader.<LoadAsync>d__4.MoveNext() in ...\OpenAPI.NET\src\Microsoft.OpenApi\Reader\Services\OpenApiWorkspaceLoader.cs:line 48
   at Microsoft.OpenApi.Reader.OpenApiModelFactory.<LoadExternalRefsAsync>d__12.MoveNext() in ...\OpenAPI.NET\src\Microsoft.OpenApi\Reader\OpenApiModelFactory.cs:line 272
   at Microsoft.OpenApi.Reader.OpenApiModelFactory.<InternalLoadAsync>d__11.MoveNext() in ...\OpenAPI.NET\src\Microsoft.OpenApi\Reader\OpenApiModelFactory.cs:line 251
   at Microsoft.OpenApi.Reader.OpenApiModelFactory.<LoadAsync>d__6.MoveNext() in ...\OpenAPI.NET\src\Microsoft.OpenApi\Reader\OpenApiModelFactory.cs:line 139
   at Microsoft.OpenApi.Models.OpenApiDocument.<LoadAsync>d__67.MoveNext() in ...\OpenAPI.NET\src\Microsoft.OpenApi\Models\OpenApiDocument.cs:line 564
   at Microsoft.OpenApi.Reader.Services.OpenApiWorkspaceLoader.<LoadAsync>d__4.MoveNext() in ...\OpenAPI.NET\src\Microsoft.OpenApi\Reader\Services\OpenApiWorkspaceLoader.cs:line 49
   at Microsoft.OpenApi.Reader.OpenApiModelFactory.<LoadExternalRefsAsync>d__12.MoveNext() in ...\OpenAPI.NET\src\Microsoft.OpenApi\Reader\OpenApiModelFactory.cs:line 272
   at Microsoft.OpenApi.Reader.OpenApiModelFactory.<InternalLoadAsync>d__11.MoveNext() in ...\OpenAPI.NET\src\Microsoft.OpenApi\Reader\OpenApiModelFactory.cs:line 251
   at Microsoft.OpenApi.Reader.OpenApiModelFactory.<LoadAsync>d__6.MoveNext() in ...\OpenAPI.NET\src\Microsoft.OpenApi\Reader\OpenApiModelFactory.cs:line 139
   at Microsoft.OpenApi.Models.OpenApiDocument.<LoadAsync>d__67.MoveNext() in ...\OpenAPI.NET\src\Microsoft.OpenApi\Models\OpenApiDocument.cs:line 564
   at ReferencesToSubFolders.Program.<Main>d__0.MoveNext() in C:\Users\...\source\repos\ReferencesToSubFolders\ReferencesToSubFolders\Program.cs:line 21
   at ReferencesToSubFolders.Program.<Main>d__0.MoveNext() in C:\Users\...\source\repos\ReferencesToSubFolders\ReferencesToSubFolders\Program.cs:line 27
   at ReferencesToSubFolders.Program.<Main>(String[] args)

This exception occurs because the Uri is build from OpenApiReaderSettings.BaseUrl and appends the relative path. This construction happens in OpenApiModelFactory in the method LoadExternalRefsAsync.

// Create workspace for all documents to live in.
var baseUrl = settings.BaseUrl ?? new Uri(OpenApiConstants.BaseRegistryUri);
var openApiWorkSpace = new OpenApiWorkspace(baseUrl);

// Load this root document into the workspace
var streamLoader = new DefaultStreamLoader(settings.BaseUrl);
var workspaceLoader = new OpenApiWorkspaceLoader(openApiWorkSpace, settings.CustomExternalLoader ?? streamLoader, settings);

My expectation would be that the baseUrl is the location of the current document. This would allow to use relative references to the current file instead of the settings base url.

I wanted to start implementing a fix also for this issue, but I would like to get some understanding of the expected behavior with a few classes:

1. Should OpenApiDocument.BaseUri be the location of the current document or https://openapi.net/<guid>?
2. Should OpenApiModelFactory.LoadExternalRefsAsync reuse the existing OpenApiWorkspace as all documents should be loaded into one workspace?
3. How should a relative reference be resolved as there is no ResolveReference on the OpenApiDocument only on the OpenApiWorkspace?

[baywet]

@dldl-cmd which version of the library are you testing with?

[dldl-cmd]

@baywet v2 - I tested with latest main branch (c4bdccc)

[baywet]

Thanks for confirming!
Since you're already setup to test this and build locally, Is this something you'd like to submit a pull request for provided some guidance?

[dldl-cmd]

@baywet Yes, I would like to do a PR on this issue. I already look at some code pieces therefore I came up with the above questions.

[baywet]

To answer your questions:

1. I think this is a placeholder when the document is not being loaded from a "remote source", maybe we should put the path there instead? @MaggieKimani1 to provide more context.
2. I believe reusing the existing workspace to load additional documents is the desired design here.
3. You should be able to navigate from the document to the workspace using the workspace property? I'm not sure I understand the question.

Let us know if you have any additional comments or questions.

[dldl-cmd]

3. I think the solution could be to have a method e.g. on the OpenApiReference or the OpenApiDocument to turn the relative reference inside of a document to an absolute reference, which could then be given to OpenApiWorkspace.ResolveReference(<absolute path>).
   
   With the OpenApiReference.HostDocument property it should be possible on a reference itself.

[baywet]

I think it'd make sense, feel free to startup a PR, seeing the code will make things a bit more concrete at this point.

[darrelmiller]

Relative references should be resolved in any implementation of IStreamLoader, not in an OpenAPIDocument.