Announcement: Consolidating .NET GitHub repos

[stephentoub]

Over the next few months, we plan to consolidate several of the foundational repositories of .NET Core, including dotnet/coreclr and dotnet/corefx.

.NET’s repository structure on GitHub was initially created in a fairly fine-grained manner, with the aim of enabling runtime agility and increased developer productivity. However, this separation has led to a variety of complications for contributors and maintainers alike. For example:

• Confusion about where issues should be opened. It’s not always clear in which repo an issue should be reported, complicated by the fact that some implementations actually span repositories (e.g. a type might be exposed through a reference assembly and have its tests in corefx but actually be implemented in System.Private.CoreLib in coreclr).
• Difficulty sharing source. System.Private.CoreLib is intricately connected to the rest of the runtime, and thus has lived with the runtime code. That, however, means that although corefx has been intended as the place to share as much core library code as possible across any underlying runtime, we’ve needed to keep a significant body of code in System.Private.CoreLib, which then has meant it’s needed to be “mirrored” to each runtime repo (coreclr, corert, mono) that needs to consume it as source; similarly, corefx has similarly required access to much of that source, for example to use the same interop DllImports that are employed elsewhere in the runtimes. The automatic mirror that shares this source needs to be maintained, and even when it has been, it’s often led to lag. It’s also complicated developer processes, where to validate a change in one repo generally requires manually mirroring the code to other local repos in order to validate the change and ensure merging into one repo won’t break the others.
• PRs spanning multiple repos. Because the runtime, CoreLib, and the core libraries are all intricately linked, changing runtime behaviors, adding new APIs, or changing various build processes (e.g. improving static analysis) often requires multiple PRs carefully staged across time and multiple repos. For example, to add a new method to a type like Dictionary<>, a developer must first make the source changes in her local coreclr repo and the test and reference assembly changes in her local corefx repo. When satisfied with the fix, the developer must submit a PR to coreclr as well as a PR to corefx, the latter of which will fail. Eventually the coreclr PR will be merged, and the changes will mirror to the other runtimes that also require the update. Eventually those mirror PRs will be merged, and builds will be created containing the fix for each runtime. At some point later those builds will be consumed into the corefx repo, after which point the original corefx PR can be re-validated and eventually merged. That’s the best case; things get more complicated in situations where there are bidirectional dependencies.
• Building an installable runtime. Self-hosting a custom built .NET requires intricate knowledge of how all the repos work and interact. A developer can’t just clone a single repo, make a desired change, and easily produce an installer.
• Consistency. One goal we had for more fine-grained repos was to enable isolation and independence for teams working within each repo’s confines, but this has led to non-trivial duplication of effort on things like build systems and CI, and then the resulting lack of consistency as each system ends up diverging from the others.

The issues go beyond the runtime. For example, the ASP.NET maintainers and community did a great job in the past year or so consolidating from ~55 repos down to ~5 repos, but that’s still more repos than is desirable, leading to similar issues as cited above for the runtime. On top of that, these ASP.NET repositories are in the aspnet GitHub org, which adds an additional set of issues, for example:

• Complications moving issues between repos. If a developer opens an issue in aspnet/aspnetcore and it’s determined that the cause of the issue is actually in dotnet/corefx, there is no GitHub mechanism to enable easily moving that issue across the aspnet to dotnet organizational boundary.
• Permissions. Each organization’s permissions end up needing to be managed and maintained separately.

The issues extend into tooling as well. For example, we currently have multiple repositories that all logically make up the dotnet CLI, but actually creating a working installer spans multiple repos.

Plan

To address these issues, we’re planning to make some changes to our repository structure:

• dotnet/platform. We plan to combine dotnet/coreclr, dotnet/corefx, dotnet/corert, dotnet/core-setup, and the relevant portions of mono/mono into a new dotnet/platform repo. Everything needed to build and produce the Microsoft.NETCore.App shared framework will be in this repo. We will no longer suffer from the complications of source mirroring. Features like changing runtime behavior or adding an API will no longer require a complicated dance across multiple repos. Etc.
• dotnet/aspnetcore. We plan to move the existing aspnet/aspnetcore repository into the dotnet organization. Along with that, we aim to combine a variety of the other aspnet repositories into aspnetcore, such as aspnet/blazor. Whether repos like entityframeworkcore remain separate or combined remains an open question. A goal is that, just as dotnet/platform will be responsible for the creation of Microsoft.NETCore.App, dotnet/aspnetcore will be responsible for the creation of Microsoft.AspNETCore.App.
• dotnet/cli. We plan to combine dotnet/toolset and dotnet/sdk into the dotnet/cli repo.

FAQ

Why are we merging repos?

We believe we can significantly improve several aspects of .NET on GitHub by combining repos, benefiting both maintainers and contributors. These improvements will manifest in a variety of ways, such as in better issue management, much easier models of contribution, and easier and faster ways to build and install the resulting bits.

What will happen to the aspnet org?

The aspnet org as a separate entity is legacy and artificial. In time we hope to absorb it into the dotnet org and sunset the aspnet org.

Will the coreclr and corefx repos go away?

With the help of GitHub, we plan to migrate all issues from these repos to the new dotnet/platform repo, and then lock down creation of additional issues, such that the repos will no longer be used for issue management. We will also inhibit the creation of PRs to the master branch, which will effectively become an archive for read-only review of history. In this sense, these repos will be archived, and no active development will happen with them. However, we plan to continue servicing previous .NET Core releases out of these repos, so the various release branches (e.g. release/2.1, release/3.0, etc.) will continue to see (limited) activity.

Will the mono repo go away?

No. mono/mono contains the source for the full mono stack and will continue to live on happily. We will simply be moving the managed source associated with System.Private.CoreLib and copying the relatively small amount of source that makes up the native mono runtime to dotnet/platform. We may choose to then use some mirroring technology to keep the runtime copy in sync (this will, however, not suffer from the same mirroring issues we currently experience, as we would not be mirroring between two components required to build the same binaries), or we may choose to let them diverge and manually sync only those changes deemed relevant to both implementations.

Will the corert repo go away?

Yes. We plan to retire/archive the corert repo. Some of the technology in the corert repo will be migrated to the master branch of dotnet/platform, where it will be productized as part of .NET. Other portions of the corert repo will be migrated to feature branches of dotnet/platform, where the experimentation can continue. In this way, we will use feature branches to continue experimenting with the corert technology, while making it easier to share portions with its shipping counterpart and also graduate functionality into master if/when it’s ready.

Does this mean there will be a single repo for all of .NET?

No. We will be reducing the number of repos that contribute to .NET, but currently we do not believe that going all the way down to one is the right answer.

Doesn’t this mean that issue and PR tracking will now be overwhelming?

It is already the case that the vast majority of issues in a given repo are not relevant to any individual developer, and with several thousand open issues in each repo and on the order of a hundred open PRs in each, we already need systems (e.g. labels) to successfully manage issues and PRs. As such, we don’t believe the merging will have a significant impact on this aspect of developer productivity. If it turns out to have an unexpectedly large negative impact, we will work with the community to find ways to mitigate the problem. However, there are already a multitude of successful open source projects on GitHub with at least an order of magnitude more issues.

What will happen to existing issues?

With the help of GitHub functionality, we plan to migrate all issues from old repos (e.g. dotnet/coreclr, dotnet/corefx) to the new repos (e.g. dotnet/platform). We may also use this as a forcing function to revisit stale issues and either close those that are no longer relevant or reinvigorate those that are demanding of more immediate attention.

What about git history?

In general, we plan to keep history, such that history from each constituent repo will be a part of the new repo. However, we have made some mistakes in the past (e.g. large binaries, multitudes of automated PRs for flowing bits and source between repos, etc.), and we plan to rewrite history to correct those mistakes wherever possible and impactful. Some rough calculations suggest this could end up significantly reducing the size of the repos as well as the time it takes to clone, which should not only help developers approaching the project but also CI. This will end up meaning that SHAs may be different in the new repo than they were in the old repo; as previously mentioned, however, any references to the old SHAs in the old repos will continue to work, as those repos will remain accessible.

Will I still show up in the contributor list due to previous contributions?

Yes. We plan to merge all such history.

Will this break debugging with SourceLink?

No. The existing repos will continue to be accessible, and commit SHAs there will remain unchanged.

Might plans change?

Sure. Part of the goal of posting this announcement is to hear from you, hear about additional benefits you're excited about, and hear about additional concerns we may not have considered. We will include such ideas in our planning and course correct as necessary.

[Eilon]

@stephentoub , feel free to get in touch with me about merging repos and some issues we ran into in the past regarding it when we merged most of the ASP.NET Core repos into mostly one repo. We learned a lot of right things to do and also a few wrong things (or, at least, less-than-ideal things).

[xsoheilalizadeh]

Bring the Xamarin to dotnet organization dotnet/xamarin. Xamarin is part of dotnet ecosystem and I don't see any reason to keeping it out of the organization. Recently its documentations were out of the Microsoft Docs but now we have it in Microsoft Docs.
According to .NET 5 Introduction

We will be able to use it to target Windows, Linux, macOS, iOS, Android, tvOS, watchOS and WebAssembly and more.

[adamralph]

Sounds great!

BTW - I've also felt these pain points in other scenarios:

• NuGet/NuGet.Client -> dotnet/cli
• microsoft/vstest -> dotnet/cli

In both of these scenarios, as I understand it, the former ships to the latter to provide dotnet commands. Is there any plan to expand the merging to include other repos, such as these, which also ship into the .NET CLI?

[KodrAus]

This looks like a great path forward to me! Would there also be any benefit in pulling in some of the LLILC pieces that CoreCLR tools and CoreRT depend on?

The only thing I’m wondering is what impact putting CoreRT components behind branches would have on their discoverability. I guess it just depends on what shape the platform repo ends up taking.

[Suchiman]

Other portions of the corert repo will be migrated to feature branches of dotnet/platform, where the experimentation can continue.

Which parts of CoreRT? I suppose the parts you want to pull into master are CPAOT (aka ILCompiler.ReadyToRun) for AOTv3, why not the rest as well? The rest of CoreRT doesn't really affect the other parts of the repo. Putting CoreRT into an experimental branch just replaces continous mirroring by continous merging from master.

[NickCraver]

I get all the moves - it sounds massive and does have downsides with notifications and such, but I do see the net win.

On the downsides: we’re working on major Razor contributions (in ASP.NET) that were interrupted the first time with the major repo combination - I’m not looking forward to sinking time into that move again. Please be aware of the community fatigue on those repos specifically. We’ll probably just forgo contributing anything significant there in 2019 and wait for things to settle.

My main question was actually dotnet/cli. While that repo name makes sense today, I can’t see it making sense with SDK and such in there. When I’m troubleshooting an MSBuild/Visual Studio issue I would not be looking at the cli repo to file an issue.

Can you please share some rational in choosing cli as the combined name? With the current info, It doesn’t seem like it sounds accurate for what’s planned or (more importantly, IMO) intuitive for the future. I would think dotnet/toolchain or something may make more sense...so tossing that out there.

[gulbanana]

alternatively, a merge from cli -> sdk instead of sdk -> cli. it follows common usage, and after all, the sdk is what you install to get a copy of the cli...

[pebezo]

dotnet/sdk or dotnet/toolchain makes more sense to me too (prefer sdk). While you're at it, why not use dotnet/asp instead of dotnet/aspnetcore (the Core term is getting deprecated and .net is already in dotnet)

[christianscheuer]

Can you please elaborate on the plans for CoreRT - ie. what will continue to work on these suggested feature branches? May we continue to use the native compilation on our existing projects that depend on it via the same features that work today? Can we continue to rely on existing published packages at the nuget feed or do we risk that our projects suddenly stop compiling?

[mjsabby]

Please don't put CoreRT in a separate branch. Why can't it be put in a folder called experimental?

[stephentoub]

@jkotas can best speak to corert. corert today doesn't contribute to .NET Core and is only tangentially related to this consolidation effort for the repos that do compose the shipping .NET. I included it in the description to try to make things clearer, but it seems to have had the opposite effect; apologies. The mentions of corert were meant to highlight that there are pieces that we plan to consume into coreclr moving forward, which would mean they'd end up in whatever branch coreclr was actively being maintained in. Everything else about the corert experiment would remain separate in some fashion.

[stephentoub]

@nguerrera, can you speak to the questions about dotnet/cli?

[stephentoub]

feel free to get in touch with me about merging repos and some issues we ran into in the past regarding it when we merged most of the ASP.NET Core repos into mostly one repo. We learned a lot of right things to do and also a few wrong things (or, at least, less-than-ideal things).

@Eilon, thanks. Nate shared his experiences from the effort, the good and the less-than-ideal, but yours would be welcome as well.

[nguerrera]

It seems the main concern is about the name for CLI. I think we would be open to another name for the merger of (cli + sdk + toolset).

I think we want to avoid 'sdk' because people confuse it with 'core-sdk', which is actually the whole SDK box coming together, including the runtimes. That said, it does produce the sdk\(version) folder underneath dotnet.

'toolset' is actually relatively accurate for the combination, and what we named the top repo of these 3, so maybe we use that. I've seen a few votes for toolchain above. @livarcocc , what do you think?

[NickCraver]

An issue that came up in Twitter discussions was one I didn’t think of: coding style/convention differences between coreclr and corefx today. We point people at corefx as an example of what to do and coreclr...not so much. What is the plan there now that the boundary disappears?

@nguerrera toolset or toolchain both work well IMO - if toolset is easier then 👍

[stephentoub]

An issue that came up in Twitter discussions was one I didn’t think of: coding style/convention differences between coreclr and corefx today. We point people at corefx as an example of what to do and coreclr...not so much. What is the plan there now that the boundary disappears?

The style employed by corefx. We've already started cleaning up Corelib, adding StyleCop, adding FxCop analyzers, fixing a bunch of stuff, etc.

[forestbat]

@stephentoub #12884 See this. Will structure of dotnet's packages have change？

[PathogenDavid]

As far as consolidating issues go, I hope the existing boundaries are preserved through a distinct set of labels.

I agree that it's currently very confusing for newcomers where an issue should best be filed, but I think the separation is valuable for people who are more experienced with the ecosystem.

For instance, when originally researching https://github.com/dotnet/coreclr/issues/23168 I knew that CoreCLR was going to be the most applicable repo so it's where I focused my attention. Searching things relating to stack traces in corefx would've yielded thousands of unrelated issues.

The current area-Xyz labels are too granular and numerous to help in this regard. In a theoretical world where everything is merged as-is with no new labels, there's literally hundreds of area-System.Whatever labels I'd want to exclude to ignore framework stuff. On the flip side, it wouldn't be immediately obvious which area label from CoreCLR is the most relevant. My issue got labeled area-Diagnostics, but I could also see area-Interop and area-ExceptionHandling also being relevant and it could take quite some time to go through the hundreds of labels to discern what is/isn't relevant. You could search broad and refine down based on the results, but for such a niche issue you might miss something important.

I'm sure this is planned, but wanted to mention it just in case.

TL;DR: Please make component-CoreFX, component-CoreCLR, etc labels.

[PathogenDavid]

The only thing I’m wondering is what impact putting CoreRT components behind branches would have on their discoverability. I guess it just depends on what shape the platform repo ends up taking.

I too share this concern about CoreRT. Especially when it comes to issues. Right now it's very easy to look at https://github.com/dotnet/corert/issues and get an idea of the issues affecting CoreRT. In a consolidated future, the relatively few CoreRT issues would be drowned out by CoreFX and CoreCLR issues. Labels don't help here as much because only Microsoft employees can assign labels to issues, I'd worry that the relatively-low-priority CoreRT issues would generally not get labeled in a timely manner.

I was going to say that I feel like it'd be better to have CoreRT live as a fork (in the GitHub sense of the term) of the dotnet/platform repo, but it seems GitHub does not allow a fork of a repository to exist within the same organization as the upstream.

An alternative solution would be to make a fork (in the Git sense of the term), this would allow better visibility and have a separate set of issues. A downside here is that GitHub would not recognize the repo as a fork so it'd come up in site-wide searched when dotnet/platform might be more relevant, the "this repo is a fork" thing would be missing, and the metrics stating how far ahead/behind corert-platform/master is compared to platform/master would be missing.

Another solution I'd propose is having a meta project for CoreRT issues. (Similar to dotnet/announcements.) This might seem like a step back since it brings us into the old issue of people not knowing where issues go, but I'd argue the current consumers of CoreRT are very familiar with what it is and would be willing to deal with the separate project for issues.

One final solution: We could have a convention of prefixing issues with the relevant component. (IE: Titling an issue [CoreRT] Does not work with System.Data.SqlClient) and a bot adds the appropriate label removes the prefix from the title. This would allow informed .NET issue-filers to suggest where their issues should go without opening up free-for-all label usage.

Anyway, sorry for the wall of text. Just wanted to highlight my concern and throw out some ideas that came to mind.

[ghuntley]

Sharing a tool that will make your life easier: https://github.com/google/copybara

Copybara is a tool used internally at Google. It transforms and moves code between repositories.

Often, source code needs to exist in multiple repositories, and Copybara allows you to transform and move source code between these repositories. A common case is a project that involves maintaining a confidential repository and a public repository in sync.

Copybara requires you to choose one of the repositories to be the authoritative repository, so that there is always one source of truth. However, the tool allows contributions to any repository, and any repository can be used to cut a release.

The most common use case involves repetitive movement of code from one repository to another. Copybara can also be used for moving code once to a new repository.

Examples uses of Copybara include:

Importing sections of code from a confidential repository to a public repository.

Importing code from a public repository to a confidential repository.

Importing a change from a non-authoritative repository into the authoritative repository. When a change is made in the non-authoritative repository (for example, a contributor in the public repository), Copybara transforms and moves that change into the appropriate place in the authoritative repository. Any merge conflicts are dealt with in the same way as an out-of-date change within the authoritative repository.

[Alovchin91]

I would like to express my deepest concern about the future of CoreRT.

While Microsoft seems to think about this project as a non-productizable, best-effort playground, there is at least one important use case for CoreRT which none of your AOT efforts covers — single-file, zero-dependency shared libraries with DllExport's.

Those are a way to use well-known C# and .NET technologies to write extensions and plugins for apps that don't support loading .NET assemblies out of the box. There is an outstanding 3F/DllExport project out there, but it still depends on now-deprecated, non-cross-platform .NET Framework.

Unless at least this scenario is covered by the "mainstream" AOT, CoreRT repo must not go away. Also, as far as I can see, CoreRT AOT approach based on RyuJIT codegen is considered pretty mature by many devs out there, so I believe it's best to keep CoreRT repo separate.

[IanKemp]

@stephentoub I suggest reordering this announcement so that the actual repo merges (in the "Plan" section) are listed before the rationale, as that's what most people will be immediately interested in.

[khellang]

While you're at it, why not use dotnet/asp instead of dotnet/aspnetcore (the Core term is getting deprecated and .net is already in dotnet)

Agreed, something like dotnet/web would fit in with dotnet/platform and dotnet/sdk 👍

Also, what happens to aspnet/extensions? It's always been a bit weird that the Microsoft.Extensions.* APIs lived under the aspnet org.

I'd also like to echo @adamralph's mention of NuGet. Why should it stay as a separate org? It's most definitely a part of the platform now, it only makes sense to take a look at that org as well.

[petroemil]

Weighing in on many previous comments, I too think that CoreRT shouldn't be hidden on a branch. It IS very much part of the "platform", so by all means make it part of the consolidation effort, but put it in a folder or something.

Blazor (client side) is experimental too, would that be hidden on a branch as well?

The other thing that caught my eye is the consideration of merging Entity Framework into the ASP.NET repo. I don't think that's a good idea, those are 2 separate, independent libraries... EF can be used in many other environments, they should be kept strictly separated.

[khellang]

Blazor (client side) is experimental too, would that be hidden on a branch as well?

@petroemil Blazor isn't considered experimental anymore, though:

With this newest Blazor release we’re pleased to announce that Blazor is now in official preview! Blazor is no longer experimental and we are committing to ship it as a supported web UI framework including support for running client-side in the browser on WebAssembly.

[thomaslevesque]

Also, what happens to aspnet/extensions? It's always been a bit weird that the Microsoft.Extensions.* APIs lived under the aspnet org.

Agreed. They should either be in dotnet/corefx or in their own repo under the dotnet org.

[shaggygi]

@stephentoub would it be possible to pin this issue? seems like an important one coming and peeps might not see it as it moves down stream when newer issues are added. thx

[lt-gerjan]

I'd also like to echo @adamralph's mention of NuGet. Why should it stay as a separate org? It's most definitely a part of the platform now, it only makes sense to take a look at that org as well.

Yes, please kill NuGet (in a friendly manner). Put it under dotnet/toolset. Personally, NuGet has so much negative associations that I would cheer for a complete "fusion". It already is hidden via the CLI (dotnet restore, dotnet add package). Also for issues it is always difficult where to file it: NuGet org or cli?

But I understand the challenges like the feeds, gallery, website etc.

Another interesting discussion: should MSBuild (https://github.com/microsoft/msbuild) also fall under dotnet/toolset? It is also used outside of .NET right? For .NET it would be very clear imo. Also, NuGet and MSBuild (and SDK) are becoming more integrated than ever (for example Directory.Build.props)

Edit1: After reading some valid comments I should state that my main point is about the NuGet Client. From a .NET developer perspective I think we should think about 3 or 4 concepts: the platform, the language, the app model and the toolset. And hide things like NuGet, MSBuild, SDK, XML files etc. The CLI should be able to do almost everything.

Edit2: I know the ship has already sailed but I like dot.net/pkg (bit like Go) or dot.net/packages as replacement or addition to the NuGet Gallery. And of course VS should remove NuGet names. Okay, I'm dreaming

[IanKemp]

I'd also like to echo @adamralph's mention of NuGet. Why should it stay as a separate org? It's most definitely a part of the platform now, it only makes sense to take a look at that org as well.

Yes, please kill NuGet (in a friendly manner). Put it under dotnet/toolset. Personally, NuGet has so much negative associations that I would cheer for a complete "fusion". It already is hidden via the CLI (dotnet restore, dotnet add package). Also for issues it is always difficult where to file it: NuGet org or cli?

But I understand the challenges like the feeds, gallery, website etc.

Another interesting discussion: should MSBuild (https://github.com/microsoft/msbuild) also fall under dotnet/toolset? It is also used outside of .NET right? For .NET it would be very clear imo. Also, NuGet and MSBuild are becoming more integrated than ever (for example Directory.Build.props)

Unfortunately it's not totally hidden... for example you still have to do dotnet nuget push for some godforsaken reason.

[PathogenDavid]

To all the people talking about NuGet: NuGet is not strictly a .NET technology. It's definitely most strongly associated with the .NET ecosystem, but it can be and is used in other places where a package manager can be useful. In particular, Native C++ and Chocolatey come to mind.

[adamralph]

OK, thanks @nguerrera.

[nguerrera]

I've updated the README on dotnet/cli to reflect the interim state we're at now.

[gewarren]

dotnet/platform. We plan to combine dotnet/coreclr, dotnet/corefx, dotnet/corert, dotnet/core-setup, and the relevant portions of mono/mono into a new dotnet/platform repo.

Did the new repo's name get changed to dotnet/runtime instead?

[ViktorHofer]

Did the new repo's name get changed to dotnet/runtime instead?

Yes: https://github.com/dotnet/runtime

[mairaw]

This file should be updated to reflect the new repos:
https://github.com/dotnet/core/blob/master/Documentation/core-repos.md

It's linked from https://github.com/dotnet/core#getting-help.

[stephentoub]

dotnet/corefx, dotnet/coreclr, dotnet/core-setup, and the relevant code from mono/mono have now all been consolidated into dotnet/runtime.

[9034725985]

Hi, can you please take a look at Github Actions and guide us what change we need to make?

actions/setup-dotnet#80

[danmoseley]

@9034725985 can you please open a new issue.

[ulrichb]

Will this break debugging with SourceLink?

No. The existing repos will continue to be accessible, and commit SHAs there will remain unchanged.

The old aspnet/AspnetCore repo seems to have disappeared. 😕 I'm getting 404s for their source URLs (coming from the .PDBs). For example:

https://raw.githubusercontent.com/aspnet/AspnetCore/28cf04843f3456a72f1816e83a1f4aa59f49a049/src/Microsoft.AspNetCore/WebHost.cs

[stephentoub]

The aspnet/aspnetcore repo simply moved orgs to dotnet/aspnetcore.
https://help.github.com/en/github/administering-a-repository/transferring-a-repository

My understanding is GitHub should have preserved links appropriately. From the above docs:
"All links to the previous repository location are automatically redirected to the new location."

It sounds like that didn't happen in all cases.

cc: @Pilchie, @ViktorHofer

[ViktorHofer]

FWIW I wasn't part of the aspnetcore migration but here's what I noticed:

https://raw.githubusercontent.com/aspnet/AspnetCore/28cf04843f3456a72f1816e83a1f4aa59f49a049/src/Microsoft.AspNetCore/WebHost.cs

Interesting, the URL doesn't resolve either for dotnet/aspnetcore: https://raw.githubusercontent.com/dotnet/aspnetcore/28cf04843f3456a72f1816e83a1f4aa59f49a049/src/Microsoft.AspNetCore/WebHost.cs.

I can't find a file under the path src/Microsfot.AspNetCore/WebHost.cs in commit 550435373bee800438f3510beef895ed9b46e029 which makes me wonder if that ever worked. Looks like the PDB points to an incorrect URL.

cc @Pilchie @dougbu

[stephentoub]

Looks like the PDB points to an incorrect URL

Based on the files that do exist, presumably it should have been:
https://raw.githubusercontent.com/aspnet/AspnetCore/28cf04843f3456a72f1816e83a1f4aa59f49a049/src/MetaPackages/src/Microsoft.AspNetCore/WebHost.cs
i.e. the original link was missing the "src/MetaPackages" part of the path? Regardless, the above resolves even with "aspnet/AspnetCore", so as @ViktorHofer cited, this doesn't appear to be related to the repo migration (but maybe there's a separate issue with PDB generation).

[Pilchie]

Hmm. Also tagging @tmat. Maybe we have something wrong with our SourceLink config, so it is pulling stuff relative to the project being built instead of the repo root?

[ulrichb]

Okay, it also seems to be a problem of this specific package. (For example IMiddleware or IConfigurationSource resolves fine.)

Just for reference, I'm talking about Microsoft.AspNetCore v2.1.7 and the used PDB is http://msdl.microsoft.com/download/symbols/microsoft.aspnetcore.pdb/6f8e3d27a142426faf0d0b876ed8d5bbFFFFFFFF/microsoft.aspnetcore.pdb.

[Pilchie]

Ah. 2.1 :-/

2.1 servicing got consolidated from all the other aspnet/* repos at some point, and I could totally believe that sourcelink wasn't correctly configured as part of that.

I've filed an issue on dotnet/aspnetcore to investigate.

[wtgodbe]

@ulrichb that file has since moved: https://github.com/dotnet/aspnetcore/blob/release/2.1/src/DefaultBuilder/src/WebHost.cs. Could you try using a newer PDB? The most recent version of 2.1 is 2.1.18

[ulrichb]

@wtgodbe Hmm ... 2.1.7 is the latest 2.1.* version, see https://www.nuget.org/packages/Microsoft.AspNetCore/2.1.7 .. isn't it?

[wtgodbe]

@ulrichb that is the last time we produced that particular package, but we've continued to ship releases of the 2.1 product. Our most recent was last month - see the release notes here, which has links to download the asp.net runtime: https://github.com/dotnet/aspnetcore/releases/tag/v2.1.18

[ulrichb]

Okay, but even the Microsoft.AspNetCore.App v2.1.18 package references Microsoft.AspNetCore (>= 2.1.7 && < 2.2.0) (which directly contains the WebHost class, part of the "broken" .PDB), ... so maybe the problem is that the Microsoft.AspNetCore package has never been "re-released"?

[ulrichb]

... and just guessing: Maybe it hasn't been released/updated after the directory move because the assembly's content hasn't changed (it's a pretty small assembly), but only the .PDB (paths) have changed.

[wtgodbe]

Ah, you may be on to something there. @dougbu @Pilchie perhaps we need to ship Microsoft.AspNetCore again out of 2.1 to resolve the issue w/ the .pdb

[dougbu]

Any later code movement and the directory consolidation are red herrings for this part of the discussion.

https://raw.githubusercontent.com/aspnet/AspnetCore/28cf04843f3456a72f1816e83a1f4aa59f49a049/src/MetaPackages/src/Microsoft.AspNetCore/WebHost.cs is a completely valid URI that matches what https://github.com/dotnet/aspnetcore/blob/v2.1.7/src/MetaPackages/src/Microsoft.AspNetCore/WebHost.cs shows. The PDB is messed up.

Questions:

1. Is this a general issue w/ our 2.1.x releases all up?

• Is it still a problem when we build the 'release/2.1' branch?
• If not, when did we fix it?

2. If this is a current issue, should we release everything next time?
3. Otherwise, how much needs to be released next time?

[dougbu]

@wtgodbe do we have a dotnet/aspnetcore issue tracking the sourcelink problem? This discussion is pretty far from this issue and dotnet/runtime (which doesn't even contain 2.1 code 😃) in particular.

[wtgodbe]

dotnet/aspnetcore#22469. I'll look more into it when I get some time