Skip to content

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

New issue

Have a question about this project? # for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “#”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? # to your account

Ease understandability of incremental generators #67745

Closed
Youssef1313 opened this issue Apr 11, 2023 · 6 comments
Closed

Ease understandability of incremental generators #67745

Youssef1313 opened this issue Apr 11, 2023 · 6 comments
Labels
Area-Compilers Feature - Source Generators Source Generators untriaged Issues and PRs which have not yet been triaged by a lead

Comments

@Youssef1313
Copy link
Member

I don't have a concrete proposal, but sharing thoughts if the team can come up with a good solution.

There are three groups of people:

  1. People who don't pay any attention to cache-friendliness of generators because they actually don't know. They blindly include whatever they want in the pipeline. Those end up messing things up. This will be mostly addressed by Analyzer suggestion: Improper incremental pipeline roslyn-analyzers#6352, but I believe such analyzer will have false positives. In this case, it will be confusing and hard to confirm how things will go in practice. This can be further improved by having a way for testing IncrementalStepRunReason for each step.
  2. People (like me) who are aware of the importance of cache-friendly generators, but no deep knowledge regarding internals of IIncrementalGenerator and so it's hard to judge whether a pipeline is good or not. For example, is it always bad to do step.Combine(context.CompilationProvider)? For 99.99% of the cases, it is bad. But is it always? probably no. Are we able to tell the 0.01% case? No.
  3. People who have deep knowledge of the internals. So far, I think this group only includes members of Roslyn team.

The request here is to have some way that helps understand what is happening under the hood while a generator is still in development. For example, get a string representation of the internal tables or see what is stored internally, etc.

As said earlier, I don't have a concrete proposal, and I don't know if this is actionable. But it would be really great if one can easily get an idea of what is happening underneath the hoods and be able to try different approaches and see how they will differ in practice.

This could be new APIs on the compiler side or an IDE visualization feature that can understand the generator pipeline and provide some visual explanation of how it will behave internally.

@dotnet-issue-labeler dotnet-issue-labeler bot added Area-Compilers untriaged Issues and PRs which have not yet been triaged by a lead labels Apr 11, 2023
@sharwell
Copy link
Member

sharwell commented Apr 11, 2023

As a teaser, here's a sample diagram produced by a helper we're working on. In this case, the diagram shows RazorSourceGenerator behavior following a change to a single .razor file:

Online FlowChart & Diagrams Editor - Mermaid Live Editor

@Youssef1313
Copy link
Member Author

@sharwell Wow! That is very amazing and promising.

  • Could this be made available out-of-the-box in the IDE or as a standalone tool?
  • I think a diagram like that, in addition to IncrementalStepRunReason will well-define the generator behavior, right? Both are important but will be serving different purposes, I think?

@sharwell
Copy link
Member

Could this be made available out-of-the-box in the IDE or as a standalone tool?

The current intent is to include it as a utility in roslyn-sdk's source generator testing library.

I think a diagram like that, in addition to IncrementalStepRunReason will well-define the generator behavior, right? Both are important but will be serving different purposes, I think?

IncrementalStepRunReason is used for color coding nodes in the diagram. Each node output has a class (:::ClassName syntax) applied to it indicating the reason.

@Youssef1313
Copy link
Member Author

Thanks @sharwell !

I really can't wait to see this live and try it out!

@sharwell
Copy link
Member

sharwell commented Apr 11, 2023

Mermaid source for diagram
flowchart LR
  subgraph 0 ["SourceOutput[0]"]
    direction LR
    subgraph 0_I [" "]
      direction TB
      0_I_0[I0]:::Cached
    end
    subgraph 0_O [" "]
      direction TB
      0_O_0[O0]:::Cached
    end
    0_I:::Inputs ~~~ 0_O:::Outputs
  end
  0:::Skipped
  subgraph 1 ["SourceOutput[1]"]
    direction LR
    subgraph 1_I [" "]
      direction TB
      1_I_0[I0]:::Cached
    end
    subgraph 1_O [" "]
      direction TB
      1_O_0[O0]:::Cached
    end
    1_I:::Inputs ~~~ 1_O:::Outputs
  end
  1:::Skipped
  subgraph 2 ["SourceOutput[2]<br/>0ms"]
    direction LR
    subgraph 2_I [" "]
      direction TB
      2_I_0[I0]:::Modified
    end
    subgraph 2_O [" "]
      direction TB
      2_O_0[O0]:::New
    end
    2_I:::Inputs ~~~ 2_O:::Outputs
  end
  2:::SourceOutput
  subgraph 3 ["SourceOutput[3]"]
    direction LR
    subgraph 3_I [" "]
      direction TB
      3_I_0[I0]:::Cached
    end
    subgraph 3_O [" "]
      direction TB
      3_O_0[O0]:::Cached
    end
    3_I:::Inputs ~~~ 3_O:::Outputs
  end
  3:::Skipped
  subgraph 4 ["SourceOutput[4]<br/>0ms"]
    direction LR
    subgraph 4_I [" "]
      direction TB
      4_I_0[I0]:::Modified
    end
    subgraph 4_O [" "]
      direction TB
      4_O_0[O0]:::New
    end
    4_I:::Inputs ~~~ 4_O:::Outputs
  end
  4:::SourceOutput
  subgraph 5 ["Compilation[0]"]
    direction LR
    subgraph 5_I [" "]
      direction TB
    end
    subgraph 5_O [" "]
      direction TB
      5_O_0[Compilation<br/>'TestProject']:::Cached
    end
    5_I:::Inputs ~~~ 5_O:::Outputs
  end
  5:::Context
  subgraph 6 ["WhereNotSuppressed[0]"]
    direction LR
    subgraph 6_I [" "]
      direction TB
      6_I_0[I0]:::Cached
    end
    subgraph 6_O [" "]
      direction TB
      6_O_0[O0]:::Cached
    end
    6_I:::Inputs ~~~ 6_O:::Outputs
  end
  6:::Skipped
  subgraph 7 ["WhereNotSuppressed[1]<br/>0ms"]
    direction LR
    subgraph 7_I [" "]
      direction TB
      7_I_0[I0]:::Modified
    end
    subgraph 7_O [" "]
      direction TB
      7_O_0[O0]:::Modified
    end
    7_I:::Inputs ~~~ 7_O:::Outputs
  end
  7:::Executed
  subgraph 8 ["allTagHelpers[0]"]
    direction LR
    subgraph 8_I [" "]
      direction TB
      8_I_0[I0]:::Cached
    end
    subgraph 8_O [" "]
      direction TB
      8_O_0[O0]:::Cached
    end
    8_I:::Inputs ~~~ 8_O:::Outputs
  end
  8:::Skipped
  subgraph 9 ["tagHelpersFromCompilation[0]<br/>10ms"]
    direction LR
    subgraph 9_I [" "]
      direction TB
      9_I_0[I0]:::Modified
    end
    subgraph 9_O [" "]
      direction TB
      9_O_0[O0]:::Unchanged
    end
    9_I:::Inputs ~~~ 9_O:::Outputs
  end
  9:::Executed
  subgraph 10 ["generatedDeclarationCode[0]"]
    direction LR
    subgraph 10_I [" "]
      direction TB
      10_I_0[I0]:::Cached
    end
    subgraph 10_O [" "]
      direction TB
      10_O_0[string]:::Cached
    end
    10_I:::Inputs ~~~ 10_O:::Outputs
  end
  10:::Skipped
  subgraph 11 ["generatedDeclarationCode[1]<br/>50ms"]
    direction LR
    subgraph 11_I [" "]
      direction TB
      11_I_0[I0]:::Modified
    end
    subgraph 11_O [" "]
      direction TB
      11_O_0[string]:::Modified
    end
    11_I:::Inputs ~~~ 11_O:::Outputs
  end
  11:::Executed
  subgraph 12 ["razorSourceGeneratorOptions[0]"]
    direction LR
    subgraph 12_I [" "]
      direction TB
      12_I_0[I0]:::Cached
    end
    subgraph 12_O [" "]
      direction TB
      12_O_0[O0]:::Cached
    end
    12_I:::Inputs ~~~ 12_O:::Outputs
  end
  12:::Skipped
  subgraph 13 ["sourceItemsWithDiagnostics[0]"]
    direction LR
    subgraph 13_I [" "]
      direction TB
      13_I_0[I0]:::Cached
    end
    subgraph 13_O [" "]
      direction TB
      13_O_0[O0]:::Cached
    end
    13_I:::Inputs ~~~ 13_O:::Outputs
  end
  13:::Skipped
  subgraph 14 ["sourceItemsWithDiagnostics[1]<br/>0ms"]
    direction LR
    subgraph 14_I [" "]
      direction TB
      14_I_0[I0]:::Modified
    end
    subgraph 14_O [" "]
      direction TB
      14_O_0[O0]:::Modified
    end
    14_I:::Inputs ~~~ 14_O:::Outputs
  end
  14:::Executed
  subgraph 15 ["razorSourceGeneratorOptionsAndDiagnostics[0]"]
    direction LR
    subgraph 15_I [" "]
      direction TB
      15_I_0[I0]:::Cached
    end
    subgraph 15_O [" "]
      direction TB
      15_O_0[O0]:::Cached
    end
    15_I:::Inputs ~~~ 15_O:::Outputs
  end
  15:::Skipped
  subgraph 16 ["AdditionalTexts[0]"]
    direction LR
    subgraph 16_I [" "]
      direction TB
    end
    subgraph 16_O [" "]
      direction TB
      16_O_0[AdditionalText<br/>'Pages/Index.razor']:::Cached
    end
    16_I:::Inputs ~~~ 16_O:::Outputs
  end
  16:::Context
  subgraph 17 ["AdditionalTexts[1]"]
    direction LR
    subgraph 17_I [" "]
      direction TB
    end
    subgraph 17_O [" "]
      direction TB
      17_O_0[AdditionalText<br/>'Pages/Counter.razor']:::Modified
    end
    17_I:::Inputs ~~~ 17_O:::Outputs
  end
  17:::Context
  subgraph 18 ["CollectedSourceItems[0]<br/>0ms"]
    direction LR
    subgraph 18_I [" "]
      direction TB
      18_I_0[I0]:::Cached ~~~ 18_I_1[I1]:::Modified
    end
    subgraph 18_O [" "]
      direction TB
      18_O_0[O0]:::Modified
    end
    18_I:::Inputs ~~~ 18_O:::Outputs
  end
  18:::Executed
  subgraph 19 ["additionalTexts[0]"]
    direction LR
    subgraph 19_I [" "]
      direction TB
      19_I_0[I0]:::Cached
    end
    subgraph 19_O [" "]
      direction TB
      19_O_0[AdditionalText<br/>'Pages/Index.razor']:::Cached
    end
    19_I:::Inputs ~~~ 19_O:::Outputs
  end
  19:::Skipped
  subgraph 20 ["additionalTexts[1]<br/>0ms"]
    direction LR
    subgraph 20_I [" "]
      direction TB
      20_I_0[I0]:::Modified
    end
    subgraph 20_O [" "]
      direction TB
      20_O_0[AdditionalText<br/>'Pages/Counter.razor']:::Modified
    end
    20_I:::Inputs ~~~ 20_O:::Outputs
  end
  20:::Executed
  subgraph 21 ["tagHelpersFromReferences[0]"]
    direction LR
    subgraph 21_I [" "]
      direction TB
      21_I_0[I0]:::Cached
    end
    subgraph 21_O [" "]
      direction TB
      21_O_0[O0]:::Cached
    end
    21_I:::Inputs ~~~ 21_O:::Outputs
  end
  21:::Skipped
  subgraph 22 ["isGeneratorSuppressed[0]"]
    direction LR
    subgraph 22_I [" "]
      direction TB
      22_I_0[I0]:::Cached
    end
    subgraph 22_O [" "]
      direction TB
      22_O_0[False]:::Cached
    end
    22_I:::Inputs ~~~ 22_O:::Outputs
  end
  22:::Skipped
  subgraph 23 ["Combine[0]"]
    direction LR
    subgraph 23_I [" "]
      direction TB
      23_I_0[I0]:::Cached ~~~ 23_I_1[I1]:::Cached
    end
    subgraph 23_O [" "]
      direction TB
      23_O_0[O0]:::Cached
    end
    23_I:::Inputs ~~~ 23_O:::Outputs
  end
  23:::Skipped
  subgraph 24 ["Combine[1]"]
    direction LR
    subgraph 24_I [" "]
      direction TB
      24_I_0[I0]:::Cached ~~~ 24_I_1[I1]:::Cached
    end
    subgraph 24_O [" "]
      direction TB
      24_O_0[O0]:::Cached
    end
    24_I:::Inputs ~~~ 24_O:::Outputs
  end
  24:::Skipped
  subgraph 25 ["Combine[2]"]
    direction LR
    subgraph 25_I [" "]
      direction TB
      25_I_0[I0]:::Cached ~~~ 25_I_1[I1]:::Cached
    end
    subgraph 25_O [" "]
      direction TB
      25_O_0[O0]:::Cached
    end
    25_I:::Inputs ~~~ 25_O:::Outputs
  end
  25:::Skipped
  subgraph 26 ["Combine[3]<br/>0ms"]
    direction LR
    subgraph 26_I [" "]
      direction TB
      26_I_0[I0]:::Modified ~~~ 26_I_1[I1]:::Cached
    end
    subgraph 26_O [" "]
      direction TB
      26_O_0[O0]:::Modified
    end
    26_I:::Inputs ~~~ 26_O:::Outputs
  end
  26:::Executed
  subgraph 27 ["Combine[4]<br/>0ms"]
    direction LR
    subgraph 27_I [" "]
      direction TB
      27_I_0[I0]:::Modified ~~~ 27_I_1[I1]:::Cached
    end
    subgraph 27_O [" "]
      direction TB
      27_O_0[O0]:::Modified
    end
    27_I:::Inputs ~~~ 27_O:::Outputs
  end
  27:::Executed
  subgraph 28 ["Combine[5]"]
    direction LR
    subgraph 28_I [" "]
      direction TB
      28_I_0[I0]:::Cached ~~~ 28_I_1[I1]:::Cached
    end
    subgraph 28_O [" "]
      direction TB
      28_O_0[O0]:::Cached
    end
    28_I:::Inputs ~~~ 28_O:::Outputs
  end
  28:::Skipped
  subgraph 29 ["Combine[6]"]
    direction LR
    subgraph 29_I [" "]
      direction TB
      29_I_0[I0]:::Cached ~~~ 29_I_1[I1]:::Cached
    end
    subgraph 29_O [" "]
      direction TB
      29_O_0[O0]:::Cached
    end
    29_I:::Inputs ~~~ 29_O:::Outputs
  end
  29:::Skipped
  subgraph 30 ["Combine[7]"]
    direction LR
    subgraph 30_I [" "]
      direction TB
      30_I_0[I0]:::Cached ~~~ 30_I_1[I1]:::Cached
    end
    subgraph 30_O [" "]
      direction TB
      30_O_0[O0]:::Cached
    end
    30_I:::Inputs ~~~ 30_O:::Outputs
  end
  30:::Skipped
  subgraph 31 ["Combine[8]"]
    direction LR
    subgraph 31_I [" "]
      direction TB
      31_I_0[I0]:::Cached ~~~ 31_I_1[I1]:::Cached
    end
    subgraph 31_O [" "]
      direction TB
      31_O_0[O0]:::Cached
    end
    31_I:::Inputs ~~~ 31_O:::Outputs
  end
  31:::Skipped
  subgraph 32 ["Combine[9]<br/>0ms"]
    direction LR
    subgraph 32_I [" "]
      direction TB
      32_I_0[I0]:::Modified ~~~ 32_I_1[I1]:::Cached
    end
    subgraph 32_O [" "]
      direction TB
      32_O_0[O0]:::Modified
    end
    32_I:::Inputs ~~~ 32_O:::Outputs
  end
  32:::Executed
  subgraph 33 ["Combine[10]<br/>0ms"]
    direction LR
    subgraph 33_I [" "]
      direction TB
      33_I_0[I0]:::Modified ~~~ 33_I_1[I1]:::Cached
    end
    subgraph 33_O [" "]
      direction TB
      33_O_0[O0]:::Modified
    end
    33_I:::Inputs ~~~ 33_O:::Outputs
  end
  33:::Executed
  subgraph 34 ["Combine[11]<br/>0ms"]
    direction LR
    subgraph 34_I [" "]
      direction TB
      34_I_0[I0]:::Modified ~~~ 34_I_1[I1]:::Cached
    end
    subgraph 34_O [" "]
      direction TB
      34_O_0[O0]:::Modified
    end
    34_I:::Inputs ~~~ 34_O:::Outputs
  end
  34:::Executed
  subgraph 35 ["Combine[12]<br/>0ms"]
    direction LR
    subgraph 35_I [" "]
      direction TB
      35_I_0[I0]:::Cached ~~~ 35_I_1[I1]:::Modified
    end
    subgraph 35_O [" "]
      direction TB
      35_O_0[O0]:::Modified
    end
    35_I:::Inputs ~~~ 35_O:::Outputs
  end
  35:::Executed
  subgraph 36 ["Combine[13]<br/>0ms"]
    direction LR
    subgraph 36_I [" "]
      direction TB
      36_I_0[I0]:::Modified ~~~ 36_I_1[I1]:::Cached
    end
    subgraph 36_O [" "]
      direction TB
      36_O_0[O0]:::Modified
    end
    36_I:::Inputs ~~~ 36_O:::Outputs
  end
  36:::Executed
  subgraph 37 ["Combine[14]"]
    direction LR
    subgraph 37_I [" "]
      direction TB
      37_I_0[I0]:::Cached ~~~ 37_I_1[I1]:::Cached
    end
    subgraph 37_O [" "]
      direction TB
      37_O_0[O0]:::Cached
    end
    37_I:::Inputs ~~~ 37_O:::Outputs
  end
  37:::Skipped
  subgraph 38 ["Combine[15]"]
    direction LR
    subgraph 38_I [" "]
      direction TB
      38_I_0[I0]:::Cached ~~~ 38_I_1[I1]:::Unchanged
    end
    subgraph 38_O [" "]
      direction TB
      38_O_0[O0]:::Cached
    end
    38_I:::Inputs ~~~ 38_O:::Outputs
  end
  38:::Skipped
  subgraph 39 ["Combine[16]"]
    direction LR
    subgraph 39_I [" "]
      direction TB
      39_I_0[I0]:::Unchanged ~~~ 39_I_1[I1]:::Cached
    end
    subgraph 39_O [" "]
      direction TB
      39_O_0[O0]:::Cached
    end
    39_I:::Inputs ~~~ 39_O:::Outputs
  end
  39:::Skipped
  subgraph 40 ["Combine[17]"]
    direction LR
    subgraph 40_I [" "]
      direction TB
      40_I_0[I0]:::Cached ~~~ 40_I_1[I1]:::Cached
    end
    subgraph 40_O [" "]
      direction TB
      40_O_0[O0]:::Cached
    end
    40_I:::Inputs ~~~ 40_O:::Outputs
  end
  40:::Skipped
  subgraph 41 ["Combine[18]"]
    direction LR
    subgraph 41_I [" "]
      direction TB
      41_I_0[I0]:::Cached ~~~ 41_I_1[I1]:::Cached
    end
    subgraph 41_O [" "]
      direction TB
      41_O_0[O0]:::Cached
    end
    41_I:::Inputs ~~~ 41_O:::Outputs
  end
  41:::Skipped
  subgraph 42 ["Combine[19]<br/>0ms"]
    direction LR
    subgraph 42_I [" "]
      direction TB
      42_I_0[I0]:::Modified ~~~ 42_I_1[I1]:::Cached
    end
    subgraph 42_O [" "]
      direction TB
      42_O_0[O0]:::Modified
    end
    42_I:::Inputs ~~~ 42_O:::Outputs
  end
  42:::Executed
  subgraph 43 ["Combine[20]<br/>0ms"]
    direction LR
    subgraph 43_I [" "]
      direction TB
      43_I_0[I0]:::Modified ~~~ 43_I_1[I1]:::Cached
    end
    subgraph 43_O [" "]
      direction TB
      43_O_0[O0]:::Modified
    end
    43_I:::Inputs ~~~ 43_O:::Outputs
  end
  43:::Executed
  subgraph 44 ["Combine[21]<br/>0ms"]
    direction LR
    subgraph 44_I [" "]
      direction TB
      44_I_0[I0]:::Modified ~~~ 44_I_1[I1]:::Cached
    end
    subgraph 44_O [" "]
      direction TB
      44_O_0[O0]:::Modified
    end
    44_I:::Inputs ~~~ 44_O:::Outputs
  end
  44:::Executed
  subgraph 45 ["Collect[0]"]
    direction LR
    subgraph 45_I [" "]
      direction TB
    end
    subgraph 45_O [" "]
      direction TB
      45_O_0[O0]:::Cached
    end
    45_I:::Inputs ~~~ 45_O:::Outputs
  end
  45:::Context
  subgraph 46 ["Collect[1]"]
    direction LR
    subgraph 46_I [" "]
      direction TB
    end
    subgraph 46_O [" "]
      direction TB
      46_O_0[O0]:::Cached
    end
    46_I:::Inputs ~~~ 46_O:::Outputs
  end
  46:::Context
  subgraph 47 ["Collect[2]<br/>0ms"]
    direction LR
    subgraph 47_I [" "]
      direction TB
      47_I_0[I0]:::Cached ~~~ 47_I_1[I1]:::Modified
    end
    subgraph 47_O [" "]
      direction TB
      47_O_0[O0]:::Modified
    end
    47_I:::Inputs ~~~ 47_O:::Outputs
  end
  47:::Executed
  subgraph 48 ["HasRazorFiles[0]<br/>0ms"]
    direction LR
    subgraph 48_I [" "]
      direction TB
      48_I_0[I0]:::Modified
    end
    subgraph 48_O [" "]
      direction TB
      48_O_0[True]:::Unchanged
    end
    48_I:::Inputs ~~~ 48_O:::Outputs
  end
  48:::Executed
  subgraph 49 ["ParseOptions[0]"]
    direction LR
    subgraph 49_I [" "]
      direction TB
    end
    subgraph 49_O [" "]
      direction TB
      49_O_0[ParseOptions]:::Cached
    end
    49_I:::Inputs ~~~ 49_O:::Outputs
  end
  49:::Context
  subgraph 50 ["Where[0]"]
    direction LR
    subgraph 50_I [" "]
      direction TB
      50_I_0[I0]:::Cached
    end
    subgraph 50_O [" "]
      direction TB
      50_O_0[AdditionalText<br/>'Pages/Index.razor']:::Cached
    end
    50_I:::Inputs ~~~ 50_O:::Outputs
  end
  50:::Skipped
  subgraph 51 ["Where[1]<br/>0ms"]
    direction LR
    subgraph 51_I [" "]
      direction TB
      51_I_0[I0]:::Modified
    end
    subgraph 51_O [" "]
      direction TB
      51_O_0[AdditionalText<br/>'Pages/Counter.razor']:::Modified
    end
    51_I:::Inputs ~~~ 51_O:::Outputs
  end
  51:::Executed
  subgraph 52 ["generatedOutput[0]"]
    direction LR
    subgraph 52_I [" "]
      direction TB
      52_I_0[I0]:::Cached
    end
    subgraph 52_O [" "]
      direction TB
      52_O_0[O0]:::Cached
    end
    52_I:::Inputs ~~~ 52_O:::Outputs
  end
  52:::Skipped
  subgraph 53 ["generatedOutput[1]<br/>11ms"]
    direction LR
    subgraph 53_I [" "]
      direction TB
      53_I_0[I0]:::Modified
    end
    subgraph 53_O [" "]
      direction TB
      53_O_0[O0]:::Modified
    end
    53_I:::Inputs ~~~ 53_O:::Outputs
  end
  53:::Executed
  subgraph 54 ["sourceItems[0]"]
    direction LR
    subgraph 54_I [" "]
      direction TB
      54_I_0[I0]:::Cached
    end
    subgraph 54_O [" "]
      direction TB
      54_O_0[O0]:::Cached
    end
    54_I:::Inputs ~~~ 54_O:::Outputs
  end
  54:::Skipped
  subgraph 55 ["sourceItems[1]<br/>0ms"]
    direction LR
    subgraph 55_I [" "]
      direction TB
      55_I_0[I0]:::Modified
    end
    subgraph 55_O [" "]
      direction TB
      55_O_0[O0]:::Modified
    end
    55_I:::Inputs ~~~ 55_O:::Outputs
  end
  55:::Executed
  subgraph 56 ["WhereValueNotNull[0]"]
    direction LR
    subgraph 56_I [" "]
      direction TB
      56_I_0[I0]:::Cached
    end
    subgraph 56_O [" "]
      direction TB
      56_O_0[O0]:::Cached
    end
    56_I:::Inputs ~~~ 56_O:::Outputs
  end
  56:::Skipped
  subgraph 57 ["WhereValueNotNull[1]<br/>0ms"]
    direction LR
    subgraph 57_I [" "]
      direction TB
      57_I_0[I0]:::Modified
    end
    subgraph 57_O [" "]
      direction TB
      57_O_0[O0]:::Modified
    end
    57_I:::Inputs ~~~ 57_O:::Outputs
  end
  57:::Executed
  subgraph 58 ["componentFiles[0]"]
    direction LR
    subgraph 58_I [" "]
      direction TB
      58_I_0[I0]:::Cached
    end
    subgraph 58_O [" "]
      direction TB
      58_O_0[O0]:::Cached
    end
    58_I:::Inputs ~~~ 58_O:::Outputs
  end
  58:::Skipped
  subgraph 59 ["componentFiles[1]<br/>0ms"]
    direction LR
    subgraph 59_I [" "]
      direction TB
      59_I_0[I0]:::Modified
    end
    subgraph 59_O [" "]
      direction TB
      59_O_0[O0]:::Modified
    end
    59_I:::Inputs ~~~ 59_O:::Outputs
  end
  59:::Executed
  subgraph 60 ["AnalyzerConfigOptions[0]"]
    direction LR
    subgraph 60_I [" "]
      direction TB
    end
    subgraph 60_O [" "]
      direction TB
      60_O_0[AnalyzerConfigOptionsProvider]:::Cached
    end
    60_I:::Inputs ~~~ 60_O:::Outputs
  end
  60:::Context
  subgraph 61 ["AnalyzerConfigOptions[1]"]
    direction LR
    subgraph 61_I [" "]
      direction TB
    end
    subgraph 61_O [" "]
      direction TB
      61_O_0[AnalyzerConfigOptionsProvider]:::Cached
    end
    61_I:::Inputs ~~~ 61_O:::Outputs
  end
  61:::Context
  subgraph 62 ["generatedDeclarationSyntaxTrees[0]"]
    direction LR
    subgraph 62_I [" "]
      direction TB
      62_I_0[I0]:::Cached
    end
    subgraph 62_O [" "]
      direction TB
      62_O_0[O0]:::Cached
    end
    62_I:::Inputs ~~~ 62_O:::Outputs
  end
  62:::Skipped
  subgraph 63 ["generatedDeclarationSyntaxTrees[1]<br/>11ms"]
    direction LR
    subgraph 63_I [" "]
      direction TB
      63_I_0[I0]:::Modified
    end
    subgraph 63_O [" "]
      direction TB
      63_O_0[O0]:::Modified
    end
    63_I:::Inputs ~~~ 63_O:::Outputs
  end
  63:::Executed
  15_O_0 --> 0_I_0
  13_O_0 --> 1_I_0
  14_O_0 --> 2_I_0
  52_O_0 --> 3_I_0
  53_O_0 --> 4_I_0
  24_O_0 --> 6_I_0
  26_O_0 --> 7_I_0
  39_O_0 --> 8_I_0
  36_O_0 --> 9_I_0
  30_O_0 --> 10_I_0
  33_O_0 --> 11_I_0
  15_O_0 --> 12_I_0
  25_O_0 --> 13_I_0
  27_O_0 --> 14_I_0
  23_O_0 --> 15_I_0
  54_O_0 --> 18_I_0
  55_O_0 --> 18_I_1
  6_O_0 --> 19_I_0
  7_O_0 --> 20_I_0
  38_O_0 --> 21_I_0
  61_O_0 --> 22_I_0
  60_O_0 --> 23_I_0
  49_O_0 --> 23_I_1
  16_O_0 --> 24_I_0
  22_O_0 --> 24_I_1
  50_O_0 --> 25_I_0
  60_O_0 --> 25_I_1
  17_O_0 --> 26_I_0
  22_O_0 --> 26_I_1
  51_O_0 --> 27_I_0
  60_O_0 --> 27_I_1
  54_O_0 --> 28_I_0
  45_O_0 --> 28_I_1
  58_O_0 --> 29_I_0
  46_O_0 --> 29_I_1
  29_O_0 --> 30_I_0
  12_O_0 --> 30_I_1
  10_O_0 --> 31_I_0
  49_O_0 --> 31_I_1
  59_O_0 --> 32_I_0
  46_O_0 --> 32_I_1
  32_O_0 --> 33_I_0
  12_O_0 --> 33_I_1
  11_O_0 --> 34_I_0
  49_O_0 --> 34_I_1
  5_O_0 --> 35_I_0
  47_O_0 --> 35_I_1
  35_O_0 --> 36_I_0
  12_O_0 --> 36_I_1
  5_O_0 --> 37_I_0
  12_O_0 --> 37_I_1
  37_O_0 --> 38_I_0
  48_O_0 --> 38_I_1
  9_O_0 --> 39_I_0
  21_O_0 --> 39_I_1
  28_O_0 --> 40_I_0
  8_O_0 --> 40_I_1
  40_O_0 --> 41_I_0
  12_O_0 --> 41_I_1
  55_O_0 --> 42_I_0
  45_O_0 --> 42_I_1
  42_O_0 --> 43_I_0
  8_O_0 --> 43_I_1
  43_O_0 --> 44_I_0
  12_O_0 --> 44_I_1
  62_O_0 --> 47_I_0
  63_O_0 --> 47_I_1
  18_O_0 --> 48_I_0
  19_O_0 --> 50_I_0
  20_O_0 --> 51_I_0
  41_O_0 --> 52_I_0
  44_O_0 --> 53_I_0
  56_O_0 --> 54_I_0
  57_O_0 --> 55_I_0
  13_O_0 --> 56_I_0
  14_O_0 --> 57_I_0
  54_O_0 --> 58_I_0
  55_O_0 --> 59_I_0
  31_O_0 --> 62_I_0
  34_O_0 --> 63_I_0
  classDef Context fill:#ccc
  classDef Skipped fill:#fffef0
  classDef SourceOutput fill:#cfc
  classDef Executed fill:#fffec0
  classDef Inputs stroke-width:0px,fill:none
  classDef Outputs stroke-width:0px,fill:none
  classDef Cached stroke-dasharray: 2 2
  classDef Modified stroke:#f00,fill:#fcc
  classDef Unchanged stroke:#00f,fill:#ccf
string ToMermaid(GeneratorRunResult)
private string ToMermaid(GeneratorRunResult result)
{
    var nodeIdCache = new Dictionary<IncrementalGeneratorRunStep, int>();

    var builder = new StringBuilder();
    builder.AppendLine("flowchart LR");

    foreach (var (name, steps) in result.TrackedOutputSteps)
    {
        for (var i = 0; i < steps.Length; i++)
        {
            var step = steps[i];
            DefineNode(step, isOutput: true, name, i, builder, nodeIdCache);
        }
    }

    foreach (var (name, steps) in result.TrackedSteps)
    {
        for (var i = 0; i < steps.Length; i++)
        {
            var step = steps[i];
            DefineNode(step, isOutput: false, name, i, builder, nodeIdCache);
        }
    }

    foreach (var step in GetAllRunSteps(result.TrackedOutputSteps.SelectMany(namedStep => namedStep.Value).Concat(result.TrackedSteps.SelectMany(namedStep => namedStep.Value))))
    {
        DefineNode(step, isOutput: false, "", 0, builder, nodeIdCache);
    }

    foreach (var (step, targetId) in nodeIdCache)
    {
        for (var inputIndex = 0; inputIndex < step.Inputs.Length; inputIndex++)
        {
            var (source, outputIndex) = step.Inputs[inputIndex];
            var sourceId = nodeIdCache[source];
            builder.AppendLine(CultureInfo.InvariantCulture, $@"  {sourceId}_O_{outputIndex} --> {targetId}_I_{inputIndex}");
        }
    }

    builder.AppendLine("  classDef Context fill:#ccc");
    builder.AppendLine("  classDef Skipped fill:#fffef0");
    builder.AppendLine("  classDef SourceOutput fill:#cfc");
    builder.AppendLine("  classDef Executed fill:#fffec0");
    builder.AppendLine("  classDef Inputs stroke-width:0px,fill:none");
    builder.AppendLine("  classDef Outputs stroke-width:0px,fill:none");
    builder.AppendLine("  classDef Cached stroke-dasharray: 2 2");
    builder.AppendLine("  classDef Modified stroke:#f00,fill:#fcc");
    builder.AppendLine("  classDef Unchanged stroke:#00f,fill:#ccf");

    return builder.ToString();

    static IEnumerable<IncrementalGeneratorRunStep> GetAllRunSteps(IEnumerable<IncrementalGeneratorRunStep> initialSteps)
    {
        var visited = new HashSet<IncrementalGeneratorRunStep>();
        var queue = new Queue<IncrementalGeneratorRunStep>(initialSteps);
        while (queue.TryDequeue(out var result))
        {
            if (!visited.Add(result))
            {
                continue;
            }

            yield return result;

            foreach (var (source, _) in result.Inputs)
            {
                queue.Enqueue(source);
            }
        }
    }

    static void DefineNode(IncrementalGeneratorRunStep step, bool isOutput, string name, int index, StringBuilder builder, Dictionary<IncrementalGeneratorRunStep, int> nodeIdCache)
    {
        if (nodeIdCache.ContainsKey(step))
        {
            return;
        }

        var id = GetNodeId(step, nodeIdCache);
        var displayName = string.IsNullOrEmpty(name) ? id.ToString(CultureInfo.InvariantCulture) : $"{name}[{index}]";
        bool includeTime = true;
        string className;
        if (!step.Inputs.Any())
        {
            className = "Context";
            includeTime = false;
        }
        else if (step.Inputs.All(step => step.Source.Outputs[step.OutputIndex].Reason is IncrementalStepRunReason.Unchanged or IncrementalStepRunReason.Cached))
        {
            className = "Skipped";
            includeTime = false;
        }
        else if (isOutput && name == "SourceOutput")
        {
            className = "SourceOutput";
        }
        else
        {
            className = "Executed";
        }

        if (includeTime)
        {
            displayName = $"{displayName}<br/>{Math.Round(step.ElapsedTime.TotalMilliseconds)}ms";
        }

        builder.AppendLine(CultureInfo.InvariantCulture, $@"  subgraph {id} [""{displayName}""]");
        builder.AppendLine(CultureInfo.InvariantCulture, $@"    direction LR");
        builder.AppendLine(CultureInfo.InvariantCulture, $@"    subgraph {id}_I ["" ""]");
        builder.AppendLine(CultureInfo.InvariantCulture, $@"      direction TB");

        if (step.Inputs.Any())
        {
            builder.Append("      ");
            for (var j = 0; j < step.Inputs.Length; j++)
            {
                if (j > 0)
                {
                    builder.Append(" ~~~ ");
                }

                builder.Append(CultureInfo.InvariantCulture, $@"{id}_I_{j}[I{j}]:::{step.Inputs[j].Source.Outputs[step.Inputs[j].OutputIndex].Reason}");
            }

            builder.AppendLine();
        }

        builder.AppendLine(CultureInfo.InvariantCulture, $@"    end");
        builder.AppendLine(CultureInfo.InvariantCulture, $@"    subgraph {id}_O ["" ""]");
        builder.AppendLine(CultureInfo.InvariantCulture, $@"      direction TB");

        if (step.Outputs.Any())
        {
            builder.Append("      ");
            for (var j = 0; j < step.Outputs.Length; j++)
            {
                if (j > 0)
                {
                    builder.Append(" ~~~ ");
                }

                builder.Append(CultureInfo.InvariantCulture, $@"{id}_O_{j}[{GetOutputName(step.Outputs[j].Value, j)}]:::{step.Outputs[j].Reason}");
            }

            builder.AppendLine();
        }

        builder.AppendLine(CultureInfo.InvariantCulture, $@"    end");
        builder.AppendLine(CultureInfo.InvariantCulture, $@"    {id}_I:::Inputs ~~~ {id}_O:::Outputs");
        builder.AppendLine(CultureInfo.InvariantCulture, $@"  end");
        builder.AppendLine(CultureInfo.InvariantCulture, $@"  {id}:::{className}");

        static string GetOutputName(object? value, int index)
        {
            return value switch
            {
                AnalyzerConfigOptionsProvider => nameof(AnalyzerConfigOptionsProvider),
                (IEnumerable, ImmutableArray<Diagnostic>) => $"O{index}",
                Compilation compilation => $"Compilation<br/>'{compilation.AssemblyName}'",
                AdditionalText text => $"AdditionalText<br/>'{text.Path}'",
                string => "string",
                bool b => b.ToString(),
                ParseOptions => nameof(ParseOptions),
                _ => $"O{index}",
            };
        }
    }

    static int GetNodeId(IncrementalGeneratorRunStep step, Dictionary<IncrementalGeneratorRunStep, int> nodeIdCache)
    {
        if (!nodeIdCache.TryGetValue(step, out var result))
        {
            result = nodeIdCache.Count;
            nodeIdCache.Add(step, result);
        }

        return result;
    }
}

@jcouv jcouv added the Feature - Source Generators Source Generators label Apr 20, 2023
@jcouv
Copy link
Member

jcouv commented Apr 20, 2023

I'm going to move this to a discussion since it lacked a concrete proposal. @sharwell If you prefer to keep this as a productivity/tooling/IDE issue, feel free to convert it back.
This graph visualization for source generators seems really interesting. Cool tool!

@dotnet dotnet locked and limited conversation to collaborators Apr 20, 2023
@jcouv jcouv converted this issue into discussion #67904 Apr 20, 2023

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

Labels
Area-Compilers Feature - Source Generators Source Generators untriaged Issues and PRs which have not yet been triaged by a lead
Projects
None yet
Development

No branches or pull requests

3 participants