Skip to content

sparkMeasure v0.19
Compare
Choose a tag to compare
@LucaCanali LucaCanali released this 10 Aug 12:26
· 95 commits to master since this release
v0.19
7cd279a

Refactoring of StageMetrics and TaskMetrics:

  • report will not use Spark SQL but rather compute the metrics using Scala code.
  • removed API for accumulables metrics capture and reporting
  • added a short report "Stages and their duration" to StageMetrics report
  • updated the Python package sparkmeasure match the Scala code refactoring

Improved documentation:

  • notably added documentation for the KafkaSink and a reference guide of sparkMeasure API

Latest release to build with Spark 2.4.8 and Scala 2.11

  • use this release if you run sparkMeasure with Spark 2.x and Scala 2.11
Assets 2
Loading