Implement Red Hat's Shenandoah GC parsing

[ghost]

Added Shenandoah GC log parsing functionality to GCViewer. Currently parsing all 6 main GC events of Shenandoah - initial mark, final mark, concurrent marking, concurrent evacuation, concurrent reset and allocation failure events. Due to concurrent evacuation, some of the statistics displayed in previous parsers are not used in Shenandoah parser.

This is a part of my BSc thesis in University of Tartu, Estonia.

EDIT: Currently supports the default -Xlog configuration, i.e. the equivalent of: -Xlog:all=warning:stderr:uptime,level,tags.

More info:
Shenandoah's OpenJDK wiki
Building latest OpenJDK9 (with Shenandoah GC)

Since it might be a hassle to set up OpenJDK just for this, added logs for testing:
shenandoah_GC_artificial_memory_leak_logs.txt
shenandoah_GC_logs.txt

[ecki]

Very cool Mart. Does it ignore datetimestamps? Maybe we need some common infrastructure for Java 9 unified GC Logs.

[chewiebug]

@bernd: should it ignore datetimestamps? And yes: The more gc algorithms with unified logging should be supported, the more need for common infrastructure / tools will arise. Probably it is best to go step by step and see what's needed, when more algorithms are added.

…

-------- Original Message -------- From: Bernd <notifications@github.com> Sent: 10 May 2017 21:58:56 CEST To: chewiebug/GCViewer <GCViewer@noreply.github.com> Cc: Subscribed <subscribed@noreply.github.com> Subject: Re: [chewiebug/GCViewer] Implement Red Hat's Shenandoah GC parsing (#187) Very cool Mart. Does it ignore datetimestamps? Maybe we need some common infrastructure for Java 9 unified GC Logs.

-- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: #187 (comment)

[ghost]

Currently supports the default configuration of Xlog, i.e. the format
[575.239s][info][gc ] GC(2243) Pause Final Mark 7934M->7671M(8192M) 52.606ms

With -Xlog::filename:timenanos datetimestamps can be displayed as
[2017-05-10T20:06:18.062+0000] GC(10) Pause Final Mark 119M->119M(128M) 0.618ms

That can be easily added, if deemed necessary.

EDIT: Here's an informative presentation at JavaOne about Unified Logging.

[shipilev]

A few comments, which might be addressed later:
a) Shenandoah also has the 8u version, the output there is more in line with other 8u GCs, not Unified Logging. I would expect 8u version to be more popular for a while;
b) Shenandoah output had changed slightly very recently with the addition of "Update References" phase, and making "Concurrent Reset Bitmaps" report the heap occupancies. See the wiki for sample output;

[codecov-io]

Codecov Report

Merging #187 into develop will increase coverage by 0.17%.
The diff coverage is 75.16%.

@@              Coverage Diff              @@
##             develop     #187      +/-   ##
=============================================
+ Coverage      62.55%   62.72%   +0.17%     
- Complexity      1264     1291      +27     
=============================================
  Files            140      142       +2     
  Lines           8057     8191     +134     
  Branches        1290     1314      +24     
=============================================
+ Hits            5040     5138      +98     
- Misses          2538     2561      +23     
- Partials         479      492      +13

Impacted Files	Coverage Δ	Complexity Δ
.../tagtraum/perf/gcviewer/imp/DataReaderFactory.java	60.68% <0%> (-1.6%)	32 <0> (ø)
...tagtraum/perf/gcviewer/view/ModelMetricsPanel.java	95.89% <0%> (-0.3%)	5 <0> (ø)
.../tagtraum/perf/gcviewer/model/AbstractGCEvent.java	85.5% <100%> (+0.61%)	41 <0> (ø)	⬇️
...gtraum/perf/gcviewer/imp/DataReaderShenandoah.java	68.18% <68.18%> (ø)	13 <13> (?)
...java/com/tagtraum/perf/gcviewer/model/GCModel.java	84.29% <78%> (-0.89%)	129 <6> (+4)
...agtraum/perf/gcviewer/model/ShenandoahGCEvent.java	94.44% <94.44%> (ø)	10 <10> (?)
...gtraum/perf/gcviewer/ctrl/impl/GcSeriesLoader.java	79.77% <0%> (-1.13%)	20% <0%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 53db4b6...a340357. Read the comment docs.

[chewiebug]

@ophillan by the way: I'll do the 1.35 release sometime during the next few weeks. After that, I'll merge your pull request.

@shipilev thank you for your comments. I will merge this pull request as it is to have preliminary support for Shenandoah. Improvements like the ones you mention can be added later.

[shipilev]

No problem with me. Ask away if you need any Shenandoah-specific questions answered.

[shipilev]

"Pause Full (System.gc())" is another frequent event.