Snap 2919 : Implementation of Structured Streaming UI Tab (#184)

* SNAP-2919 : Implementation of Structured Streaming UI Tab

Implementation of the Structured Streaming UI Tab which lets users monitor the structured streaming queries/applications statistics and progress .
Structured Streaming Tab is available both in TIBCO ComputeDB/SnappyData embedded cluster as well as in smart connector application (using Snappy Spark distribution)

Structured Streaming Tab has below capabilities:

- Listing all Structured Streaming Queries/Applications submitted to SnappyData cluster using submit-job command. Similarly in smart connector this tab will list streaming queries executed in cluster.
- Allows user selecting queries from left hand side navigation panel, to view details view on right side main query details panel.
- Query details panel displays selected queries details and statistics, as listed below;
  -- Query Name if provided, Query Id otherwise
  -- Start Date & Time
  -- Up time
  -- Trigger Interval
  -- Batches Processed
  -- Status
  -- Total Input Records
  -- Current Input Rate
  -- Current Processing Rate
  -- Total Batch Processing Time
  -- Avg. Batch Processing Time
- Query details panel also lists sources of streaming query along with each source details like type, description, input records, input and processing rate
- Query details panel also displays sink details of streaming query.
- Query details panel depicts structured streaming queries behavioural trends using following
  -- Input Records on every batch
  -- Input Rate vs Processing Rate
  -- Processing Time
  -- Aggregation State, if available
- All statistics displayed on UI are auto updated periodically 

- Adding two configurable parameters in sparks SQLConf.scala
      1) spark.sql.streaming.uiRunningQueriesDisplayLimit :
           To configure how many queries be displayed on structure streaming UI.
      2) spark.sql.streaming.uiTrendsMaxSampleSize :
           To configure how many historic data points be plotted on trends charts on structure streaming UI.

core/src/main/resources/org/apache/spark/ui/static/snappydata/snappy-commons.js

@@ -63,6 +63,73 @@ function applyNotApplicableCheck(value){
   }
 }
 
+/*
+* Utility function to convert milliseconds value in human readable
+* form Eg "2 days 14 hrs 2 mins"
+*/
+function formatDurationVerbose(ms) {
+
+  function stringify(num, unit) {
+    if (num <= 0) {
+      return "";
+    } else if (num == 1) {
+      return  num + " "+ unit;
+    } else {
+      return num + " "+ unit+'s';
+    }
+  }
+
+  var second = 1000;
+  var minute = 60 * second;
+  var hour = 60 * minute;
+  var day = 24 * hour;
+  var week = 7 * day;
+  var year = 365 * day;
+
+  var msString = "";
+  if (ms >= second && ms % second == 0) {
+    msString = "";
+  } else {
+    msString = (ms % second) + " ms";
+  }
+
+  var secString = stringify(parseInt((ms % minute) / second), "sec");
+  var minString = stringify(parseInt((ms % hour) / minute), "min");
+  var hrString = stringify(parseInt((ms % day) / hour), "hr");
+  var dayString = stringify(parseInt((ms % week) / day), "day");
+  var wkString = stringify(parseInt((ms % year) / week), "wk");
+  var yrString = stringify(parseInt(ms / year), "yr");
+
+  var finalString = msString;
+
+  if(ms >= second ) {
+    finalString = secString + " " + finalString;
+  }
+
+  if(ms >= minute ) {
+    finalString = minString + " " + finalString;
+  }
+
+  if(ms >= hour ) {
+    finalString = hrString + " " + finalString;
+  }
+
+  if(ms >= day ) {
+    finalString = dayString + " " + hrString + " " + minString;
+  }
+
+  if(ms >= week ) {
+    finalString = wkString + " " + finalString;
+  }
+
+  if(ms >= year ) {
+    finalString = yrString  + " " + wkString + " " + hrString;
+  }
+
+  return finalString;
+
+}
+
 /*
  * Utility function to convert given value in Bytes to KB or MB or GB or TB
  *

core/src/main/resources/org/apache/spark/ui/static/snappydata/snappy-streaming.css

@@ -0,0 +1,172 @@
+/* Snappy streaming CSS */
+
+#AutoUpdateErrorMsgContainer {
+  position: absolute;
+  width: 100%;
+  margin-top: -60px;
+}
+
+#AutoUpdateErrorMsg {
+  width: 30%;
+  max-height: 60px;
+  background-color: #F8DFDF;
+  border: 2px solid red;
+  border-radius: 10px;
+  z-index: 2;
+  position: relative;
+  margin: 5px auto;
+  padding: 0px 10px;
+  overflow: auto;
+  display: none;
+  text-align: center;
+  font-weight: bold;
+}
+
+.main-container {
+  width: 100%;
+  margin-top: 15px;
+}
+
+.left-navigation-panel {
+  float: left;
+  width: 15%;
+  min-width: 250px;
+  height: 100%;
+  border: solid #B1B1B1 1px;
+  background-color: #F1F1F1;
+}
+
+.right-details-panel {
+  width: 84%;
+  height: 100%;
+  float: right;
+  padding-left: 10px;
+  border: solid #B1B1B1 1px;
+  background-color: #F1F1F1;
+}
+
+.vertical-menu-heading {
+  width: 100%;
+}
+
+.vertical-menu-heading div {
+  padding: 12px;
+  font-weight: bold;
+  font-size: large;
+  text-align: center;
+  background: #A0A0A0;
+}
+
+.vertical-menu {
+  width: 100%;
+}
+
+.vertical-menu a {
+  background-color: #EEE;
+  color: black;
+  display: block;
+  padding: 12px;
+  text-decoration: none;
+}
+
+.vertical-menu a:hover {
+  background-color: #E5E5E5;
+}
+
+.vertical-menu a.active {
+  background-color: #CCC;
+  color: black;
+}
+
+.details-section {
+  text-align: center;
+  padding-left: 5px;
+  padding-right: 5px;
+}
+
+.basic-details {
+  width: 98%;
+  min-width: 250px;
+  float: left;
+  margin: 10px;
+  border: solid 2px darkgray;
+  border-radius: 10px;
+  line-height: 25px;
+}
+
+.basic-details-title {
+  float: left;
+  padding: 10px;
+  width: 50%;
+  font-size: medium;
+  font-weight: bold;
+}
+
+.basic-details > div {
+  text-align: left;
+  width: 25%;
+  float: left;
+}
+
+.basic-details-value {
+  padding: 10px;
+}
+
+.stats-block {
+  min-width: 200px;
+  width: 15%;
+  height: 100px;
+  display: inline-block;
+  margin: 10px;
+  border: solid 2px darkgray;
+  border-radius: 10px;
+}
+
+.stats-block > div {
+  margin: 10px;
+  width: auto;
+  height: 80%;
+}
+
+.stats-block-title {
+  height: 50px;
+  font-size: large;
+  font-weight: bold;
+}
+
+.stats-block-value {
+  font-size: 20px;
+}
+
+.graph-container {
+  min-width: 250px;
+  width: 23%;
+  height: 200px;
+  display: inline-block;
+  margin: 10px;
+  border: solid 1px darkgray;
+  /*box-shadow: 5px 5px 5px grey;*/
+}
+
+#selectedQueryTitle {
+  float: left;
+  margin: 10px;
+  padding: 10px;
+  font-size: 18px;
+  font-weight: bold;
+  text-align: left;
+}
+
+#selectedQueryName {
+  float: left;
+  margin: 10px;
+  padding: 10px;
+  font-size: 18px;
+  /*font-weight: bold;*/
+  text-align: left;
+}
+
+/* datatable row selection */
+table.dataTable tbody tr.queryselected {
+    background-color: #c6ccd7;
+}