Cohosting semantic tokens follow up

[davidwengier]

This is not an exhaustive TODO list for cohosting in general, but rather the items of follow up that were not done in #10097

• Refresh queue
  • Roslyn has a refresh queue that it uses on every semantic tokens request, where it queues up a refresh if the project has changed after a delay. Since the Roslyn code is now called from the OOP process, it can't do that, so we should do that on the Razor side, probably by exposing access to Roslyn's refresh queue
  • https://github.com/dotnet/roslyn/blob/711e122c86db37658d2924f2499c775ce6007b68/src/Tools/ExternalAccess/Razor/Cohost/Handlers/SemanticTokensRange.cs#L33
• FilePathService
  • Currently MEF exported from Workspaces. Needs to be abstract, with MEF exported inheritors in VS and OOP layers
  • Needs to be used in the OOP C# tokens provider. Can revert previous static method changes
  • Stop MEF exporting FilePathService from the Workspaces layer #10118
• OOP initialization
  • Currently initializes only once, which means it can't set anything that comes from the LSP client, like semantic token types. Probably need another init method that only gets called after LSP startup
  • Add proper OOP initialization for language server concepts #10119
• UsePreciseCSharpRanges
  • Needs to be implemented in OOP, and can be removed from the CSharpSemanticTokensProvider API, since it was only put there thinking LanguageServerFeatureOptions wouldn't be available, but it is
  • Remove unnecessary parameter #10120
• Telemetry
  • The cohost endpoint doesn't currently do our telemetry tracking. We can either move this down to the service, or add it to the cohost endpoint. Depends if we think we should have different telemetry properties/events (I think we should)
  • Bring telemetry back for semantic tokens in cohosting #10121
• Logging
  • Need an IRazorLoggerFactory in OOP, and set up the loghub logger in it (via the supplied TraceSource)
  • Simple logging in OOP to TraceSource #10145
• Tests
  • Are they needed? The service that does the work is the same. To follow a more Roslyn style, we would have the ISemanticTokensService do the work to switch OOP if it can, and do the work in-proc if it can't. In OOP, it then calls back into the same ISemanticTokensService, but this time there is no OOP client, and some different service impls, so it does the work in-proc. Will all of that, there would be no actual testing of OOP, just the service.

The only other things that are "todo" are doing something about IDocumentContextFactory, but that currently throws an exception, so will be fixed whenever the first usage of it is needed.

[davidwengier]

Going to close this out. There are two items above without PRs next to them, and they are:

Testing: I'm not going to bother doing anything specifically for this, because everything that actually does any work in the semantic tokens system is already fully covered by existing tests. I have been looking at how Roslyn tackles OOP testing and I have some ideas about how to write future OOP services so that we don't have to rely on "trust me bro" to know the code is shared, and covered by tests. But in this case, trust me bro.

Semantic tokens refresh: I think the bit that I had to leave out when I refactored Roslyn, I'm pretty sure is a complete no-op, just with a 2 second delay, and that the existing refresh queue is working fine and doesn't need us to do anything for it. Having said that, the Roslyn side needs to change to allow us to move to dynamic registration, so I'm incorporating those changes in my Roslyn PR (dotnet/roslyn#72609) that will set us up for that move.