Add -Vprofile option

[odersky]

Add -Vprofile and -Vprofile-sorted-by options.

This prints for each source file

• the number of code lines
• the number of tokens
• the size of serialized tasty trees (just the trees, not the name table, nor the positions)
• the complexity per line, computed by the formula (tasty-size/lines/50). That is, we
  assume 50 tasty-bytes/line as standard.

Using -Vprofile-sorted-by one can sort by a column name.

[odersky]

Two questions:

• This is currently awkwardly close in naming to the -Yprofile-enabled and related options. I believe -Yprofile-enabled is only for internal use; not sure when it was used last at all. Should we rename or remove -Yprofile-enabled?
• What is the best way to document the option and its uses? The main use would be for to pinpointing hidden complexity leading to long compile times.

[odersky]

Additionally, option -Vprofile-details N prints the details of the N most complex methods (measured by size of the generated Tasty).

[odersky]

The new format shows the Tasty size in terms of number of "chunks" where one chunk = 50 bytes. The threshold between low and moderate complexity is when chunk count and line count are the same.

[odersky]

The -Vprofile option gives some insights about code complexity and where compilation time is likely spent. If set, the compiler prints a summary for every compiled source. It looks like this:

Sourcefile                   Lines   Tokens   Tasty  Complexity/Line    Directory
ConstantOps.scala               22      210      21   0.95  low         semanticDB
LinkMode.scala                   3       20      24   8.00  high        semanticDB
Serializer.scala                28      385      32   1.14  moderate    coverage
Location.scala                  34      215      65   1.91  moderate    coverage
Coverage.scala                  19      119      93   4.89  moderate    coverage
SyntheticsExtractor.scala      118      998     112   0.95  low         semanticDB
Tools.scala                    105     1081     113   1.08  moderate    semanticDB
SemanticSymbolBuilder.scala    128     1201     141   1.10  moderate    semanticDB
Descriptor.scala               112      780     243   2.17  moderate    semanticDB
PPrint.scala                   362     3526     446   1.23  moderate    semanticDB
TypeOps.scala                  391     3740     482   1.23  moderate    semanticDB
ExtractSemanticDB.scala        410     4259     529   1.29  moderate    semanticDB
Scala3.scala                   435     4391     680   1.56  moderate    semanticDB
---------------------------------------------------------------------
Total                         2167    20925    2977   1.37  moderate    

The columns are:

• the source file name,
• the number of code lines in the source, excluding comments and whitespace,
• the number of tokens (separately parsed words) in the source,
• the number of Tasty chunks produced from the source. A chunk consists of 50 bytes of Tasty output that serializes the type-checked tree coming from the source, using some compression techniques. The size of generated Tasty is a good estimator of compile times and code complexity.
• the average complexity per line, which is computed by dividing the number of lines by the number of Tasty chunks.
• an explanation of the average complexity number, computed as follows:
  • low: fewer Tasty chunks than source lines,
  • moderate: up to 5 times as many Tasty chunks than source lines,
  • high: up to 25 times as many Tasty chunks than than source lines,
  • extreme: At least 25 times as many Tasty chunks than than source lines,
• the directory where the source is located.

The -Vprofile-sorted-by:<column label> option is like -Vprofile, but sorts the output lines according to the given column label. Possible column labels are: lines, tokens, tasty, and complexity. -Vprofile alone is equivalent to -Vprofile-sorted-by:tasty.

The -Vprofile-details N option is like -Vprofile but also summarizes the profiles of the N most complex methods (that is, the methods producing the largest Tasty output). For instance, the output of
-Vprofile-details 5 on the previously seen files will additionally print this:

Most complex methods:
Sourcefile                  Method          Lines   Tokens   Tasty  Complexity/Line    Directory
SemanticSymbolBuilder.scala addSymName         70      679      85   1.21  moderate    semanticDB
PPrint.scala                pprint             64      650      99   1.55  moderate    semanticDB
TypeOps.scala               toSemanticSig     116     1017     150   1.29  moderate    semanticDB
ExtractSemanticDB.scala     traverse          139     1455     204   1.47  moderate    semanticDB
TypeOps.scala               toSemanticType    182     1784     231   1.27  moderate    semanticDB

The -Vprofile option family is useful for trouble-spotting long compile times. For sources with low to moderate complexity one can expect compile speeds of about 5000 lines / sec on modern PCs using a warm compiler. If compile speed is significantly lower than that, one reason could be that a lot of code is not visible
in the source files, but is generated by meta programming or term synthesis from implicits. In that case,
the -Vprofile options will help identify the parts of the program that take longest to compile.

[odersky]

The previous comment is a first stab at docs for -Vprofile. The question remains where should we put these docs and when should they be shown? It basically looks like a man page, but I don't think we have man pages for options.

[dwijnand]

Well there's https://docs.scala-lang.org/overviews/compiler-options/index.html and then the various pages next to https://docs.scala-lang.org/scala3/guides/migration/options-new.html. Also, for options that take arguments, like -Wconf and -Xlint, there's a practice of adding some guidance under -Xlint:help.

[odersky]

Well there's https://docs.scala-lang.org/overviews/compiler-options/index.html and then the various pages next to https://docs.scala-lang.org/scala3/guides/migration/options-new.html. Also, for options that take arguments, like -Wconf and -Xlint, there's a practice of adding some guidance under -Xlint:help.

The closest would be to include -Vprofile in options-new.html, but then the doc would be only a line or two.
We'd need a general mechanism to find out more about an option. Ideally scalac -help -<option-name> should do something useful.

[odersky]

then the various pages next to https://docs.scala-lang.org/scala3/guides/migration/options-new.html.

I tried to add something about VProfile, but can't find that the markdown source of that file. Where is it?

[dwijnand]

https://github.com/scala/docs.scala-lang/blob/main/_overviews/scala3-migration/options-new.md

[odersky]

@dwijnand Thanks! I made a PR: scala/docs.scala-lang#2428

[som-snytt]

Current 2.13.8 example of verbose help is scalac -opt-inline-from:help (-opt is simplified for 2.13.9, -opt:inline:help).

The help arg lists choices or text. Maybe this needs a forward port.

There is a ticket to automate updating the doc page; a useful feature would be to also generate all the help texts, with a folding widget to hide them. I'll add that to the ticket, then see if that is also an easy forward port. People ask about the compiler options doc page often.

It's nice to avoid multiple options clustering around a feature, as -opt was. Possibly,

-Vprint:source,method,lines,etc     # I would expect sorting by source

with tasty default. I don't have a great trick for specifying top N. Maybe -Vprint:tasty:5. Does N mean top complexity only, or first N in requested sort order? The "double colon" sub-option I could forward port.

[odersky]

@som-snytt The problem here is that -Vprofile is not a ChoiceSetting.

[som-snytt]

The boolean check is .isSetByUser on scala 2, maybe I can find an example of choices used that way, with a default so -Vprofile works.

I don't mind trying out my suggestions as a follow-up, rather than setting in concrete here.

I'm reminded that Lukas's optimizer blog got moved to its own docs page: https://docs.scala-lang.org/overviews/compiler-options/optimizer.html

[bishabosha]

This option is not useful at all for profiling code that depend on inline methods heavily - and inlining phase can be very significant. should we enable an optional secondary pickler after inlining when -Vprofile is enabled? (otherwise what metric would we use to measure the impact of inlining?)