task4.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Hackingforfuture Task 4 - Predictive Quality Data Analysis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This predictive quality data analysis is a part of ICNAP Hackathon in Aachen on 25-27.10.2019. The task is to predict final inspection quality of a turbocharger machine. This dataset is given by IconPro GmbH."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Import Libs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import feature_engine as fe\n",
    "\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.preprocessing import MinMaxScaler\n",
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn.compose import ColumnTransformer\n",
    "\n",
    "from feature_engine.categorical_encoders import OneHotCategoricalEncoder\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "%matplotlib inline\n",
    "\n",
    "np.random.seed(48)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Loading file\n",
    "Original file is an excel file. The file is then tranformed into a csv-file first before loading into analysis.\n",
    "\n",
    "The file itself consists of datetime data (#A-#B) and three data groups - assembly data (#D - #AU), initial inspection data (#AY - #BN), and final inspection data (#BO - #CD). The assembly data stands for measured data like depth, length, force, etc. from each part of interest in the turbocharger. The inspection data (both initial and final) stands for process measurements like  gravitational force, temperature, pressure, flow, etc. as well as weight and angle of the turbocharger."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Time</th>\n",
       "      <th>Date</th>\n",
       "      <th>asm_Force #D</th>\n",
       "      <th>asm_Length #E</th>\n",
       "      <th>asm_Height #F</th>\n",
       "      <th>asm_Press-fit #G</th>\n",
       "      <th>asm_force #H</th>\n",
       "      <th>asm_Length #I</th>\n",
       "      <th>asm_Depth #J</th>\n",
       "      <th>asm_depth #K</th>\n",
       "      <th>...</th>\n",
       "      <th>fin_Tracking, Zone3 #BU</th>\n",
       "      <th>fin_Tracking, Zone4 #BV</th>\n",
       "      <th>fin_Tracking, Zone1, G #BW</th>\n",
       "      <th>fin_Tracking, Zone2, G #BX</th>\n",
       "      <th>fin_Tracking, Zone3, G #BY</th>\n",
       "      <th>fin_Tracking, Zone4, G #BZ</th>\n",
       "      <th>fin_Shape #CA</th>\n",
       "      <th>fin_pressure #CB</th>\n",
       "      <th>fin_flow #CC</th>\n",
       "      <th>fin_temperature #CD</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>02:02:09</td>\n",
       "      <td>03.01.18</td>\n",
       "      <td>0.149</td>\n",
       "      <td>102.52</td>\n",
       "      <td>0.215</td>\n",
       "      <td>12.747</td>\n",
       "      <td>1.094</td>\n",
       "      <td>135.43</td>\n",
       "      <td>1.8541</td>\n",
       "      <td>1.8517</td>\n",
       "      <td>...</td>\n",
       "      <td>96000.0</td>\n",
       "      <td>139700.0</td>\n",
       "      <td>0.14</td>\n",
       "      <td>0.12</td>\n",
       "      <td>0.07</td>\n",
       "      <td>0.08</td>\n",
       "      <td>V-shape</td>\n",
       "      <td>2.915</td>\n",
       "      <td>0.828</td>\n",
       "      <td>88.15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>02:02:41</td>\n",
       "      <td>03.01.18</td>\n",
       "      <td>0.163</td>\n",
       "      <td>102.61</td>\n",
       "      <td>0.191</td>\n",
       "      <td>12.731</td>\n",
       "      <td>1.116</td>\n",
       "      <td>135.48</td>\n",
       "      <td>1.8583</td>\n",
       "      <td>1.8556</td>\n",
       "      <td>...</td>\n",
       "      <td>97800.0</td>\n",
       "      <td>138100.0</td>\n",
       "      <td>0.16</td>\n",
       "      <td>0.14</td>\n",
       "      <td>0.06</td>\n",
       "      <td>0.12</td>\n",
       "      <td>V-shape</td>\n",
       "      <td>2.933</td>\n",
       "      <td>1.068</td>\n",
       "      <td>89.14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>02:05:06</td>\n",
       "      <td>03.01.18</td>\n",
       "      <td>0.160</td>\n",
       "      <td>102.56</td>\n",
       "      <td>0.192</td>\n",
       "      <td>12.734</td>\n",
       "      <td>1.033</td>\n",
       "      <td>135.48</td>\n",
       "      <td>1.8533</td>\n",
       "      <td>1.8487</td>\n",
       "      <td>...</td>\n",
       "      <td>112100.0</td>\n",
       "      <td>140000.0</td>\n",
       "      <td>0.04</td>\n",
       "      <td>0.03</td>\n",
       "      <td>0.02</td>\n",
       "      <td>0.15</td>\n",
       "      <td>V-shape</td>\n",
       "      <td>2.916</td>\n",
       "      <td>0.828</td>\n",
       "      <td>87.30</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>02:05:31</td>\n",
       "      <td>03.01.18</td>\n",
       "      <td>0.158</td>\n",
       "      <td>102.47</td>\n",
       "      <td>0.195</td>\n",
       "      <td>12.726</td>\n",
       "      <td>1.079</td>\n",
       "      <td>135.48</td>\n",
       "      <td>1.8578</td>\n",
       "      <td>1.8546</td>\n",
       "      <td>...</td>\n",
       "      <td>96000.0</td>\n",
       "      <td>140000.0</td>\n",
       "      <td>0.25</td>\n",
       "      <td>0.20</td>\n",
       "      <td>0.07</td>\n",
       "      <td>0.18</td>\n",
       "      <td>Ascending</td>\n",
       "      <td>2.925</td>\n",
       "      <td>1.068</td>\n",
       "      <td>88.68</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>02:08:00</td>\n",
       "      <td>03.01.18</td>\n",
       "      <td>0.154</td>\n",
       "      <td>102.61</td>\n",
       "      <td>0.207</td>\n",
       "      <td>12.752</td>\n",
       "      <td>1.061</td>\n",
       "      <td>135.48</td>\n",
       "      <td>1.8594</td>\n",
       "      <td>1.8543</td>\n",
       "      <td>...</td>\n",
       "      <td>100500.0</td>\n",
       "      <td>137700.0</td>\n",
       "      <td>0.14</td>\n",
       "      <td>0.13</td>\n",
       "      <td>0.10</td>\n",
       "      <td>0.23</td>\n",
       "      <td>Ascending</td>\n",
       "      <td>2.904</td>\n",
       "      <td>0.816</td>\n",
       "      <td>87.29</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 78 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       Time      Date  asm_Force #D  asm_Length #E  asm_Height #F  \\\n",
       "0  02:02:09  03.01.18         0.149         102.52          0.215   \n",
       "1  02:02:41  03.01.18         0.163         102.61          0.191   \n",
       "2  02:05:06  03.01.18         0.160         102.56          0.192   \n",
       "3  02:05:31  03.01.18         0.158         102.47          0.195   \n",
       "4  02:08:00  03.01.18         0.154         102.61          0.207   \n",
       "\n",
       "   asm_Press-fit #G  asm_force #H  asm_Length #I  asm_Depth #J  asm_depth #K  \\\n",
       "0            12.747         1.094         135.43        1.8541        1.8517   \n",
       "1            12.731         1.116         135.48        1.8583        1.8556   \n",
       "2            12.734         1.033         135.48        1.8533        1.8487   \n",
       "3            12.726         1.079         135.48        1.8578        1.8546   \n",
       "4            12.752         1.061         135.48        1.8594        1.8543   \n",
       "\n",
       "   ...  fin_Tracking, Zone3 #BU  fin_Tracking, Zone4 #BV  \\\n",
       "0  ...                  96000.0                 139700.0   \n",
       "1  ...                  97800.0                 138100.0   \n",
       "2  ...                 112100.0                 140000.0   \n",
       "3  ...                  96000.0                 140000.0   \n",
       "4  ...                 100500.0                 137700.0   \n",
       "\n",
       "   fin_Tracking, Zone1, G #BW  fin_Tracking, Zone2, G #BX  \\\n",
       "0                        0.14                        0.12   \n",
       "1                        0.16                        0.14   \n",
       "2                        0.04                        0.03   \n",
       "3                        0.25                        0.20   \n",
       "4                        0.14                        0.13   \n",
       "\n",
       "   fin_Tracking, Zone3, G #BY  fin_Tracking, Zone4, G #BZ  fin_Shape #CA  \\\n",
       "0                        0.07                        0.08        V-shape   \n",
       "1                        0.06                        0.12        V-shape   \n",
       "2                        0.02                        0.15        V-shape   \n",
       "3                        0.07                        0.18      Ascending   \n",
       "4                        0.10                        0.23      Ascending   \n",
       "\n",
       "   fin_pressure #CB  fin_flow #CC  fin_temperature #CD  \n",
       "0             2.915         0.828                88.15  \n",
       "1             2.933         1.068                89.14  \n",
       "2             2.916         0.828                87.30  \n",
       "3             2.925         1.068                88.68  \n",
       "4             2.904         0.816                87.29  \n",
       "\n",
       "[5 rows x 78 columns]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "filename = 'csvdata.csv'\n",
    "df = pd.read_csv(filename)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Distinguish columns\n",
    "Due to my laziness to join data table, I created one universal dataframe with prepended column names: `asm_`, `int_`, and `fin_` which stand for assembly data, initial inspection data, and final inspection data respectively."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Ti',\n",
       " 'Da',\n",
       " 'asm_Force',\n",
       " 'asm_Length',\n",
       " 'asm_Height',\n",
       " 'asm_Press-fit',\n",
       " 'asm_force',\n",
       " 'asm_Length',\n",
       " 'asm_Depth',\n",
       " 'asm_depth',\n",
       " 'asm_Depth',\n",
       " 'asm_parallelism',\n",
       " 'asm_force',\n",
       " 'asm_Length',\n",
       " 'asm_force',\n",
       " 'asm_Length',\n",
       " 'asm_torque 1',\n",
       " 'asm_angle 1',\n",
       " 'asm_torque 2',\n",
       " 'asm_angle 2',\n",
       " 'asm_torque 3',\n",
       " 'asm_angle 3',\n",
       " 'asm_torque 1',\n",
       " 'asm_torque 2',\n",
       " 'asm_torque 3',\n",
       " 'asm_angle 1',\n",
       " 'asm_angle 2',\n",
       " 'asm_angle 3',\n",
       " 'asm_torque 1',\n",
       " 'asm_torque 2',\n",
       " 'asm_torque 3',\n",
       " 'asm_torque 4',\n",
       " 'asm_angle 1',\n",
       " 'asm_angle 2',\n",
       " 'asm_angle 3',\n",
       " 'asm_angle 4',\n",
       " 'asm_temperature',\n",
       " 'asm_force',\n",
       " 'asm_Length',\n",
       " 'asm_force',\n",
       " 'asm_Temperature',\n",
       " 'asm_angle',\n",
       " 'asm_torque',\n",
       " 'asm_angle',\n",
       " 'asm_torque',\n",
       " 'asm_clearance',\n",
       " 'int_weight',\n",
       " 'int_Angle position ',\n",
       " 'int_weight',\n",
       " 'int_Angle position',\n",
       " 'int_Tracking, Zone1',\n",
       " 'int_Tracking, Zone2',\n",
       " 'int_Tracking, Zone3',\n",
       " 'int_Tracking, Zone4',\n",
       " 'int_Tracking, Zone1, G',\n",
       " 'int_Tracking, Zone2, G',\n",
       " 'int_Tracking, Zone3, G',\n",
       " 'int_Tracking, Zone4, G',\n",
       " 'int_Shape',\n",
       " 'int_pressure',\n",
       " 'int_flow',\n",
       " 'int_temperature',\n",
       " 'fin_weight',\n",
       " 'fin_Angle',\n",
       " 'fin_weight',\n",
       " 'fin_Angle',\n",
       " 'fin_Tracking, Zoe1',\n",
       " 'fin_Tracking, Zone2',\n",
       " 'fin_Tracking, Zone3',\n",
       " 'fin_Tracking, Zone4',\n",
       " 'fin_Tracking, Zone1, G',\n",
       " 'fin_Tracking, Zone2, G',\n",
       " 'fin_Tracking, Zone3, G',\n",
       " 'fin_Tracking, Zone4, G',\n",
       " 'fin_Shape',\n",
       " 'fin_pressure',\n",
       " 'fin_flow',\n",
       " 'fin_temperature']"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cols = df.columns.tolist()\n",
    "# remove # from column names\n",
    "for idc, c in enumerate(cols):\n",
    "    for idx, char in enumerate(c):\n",
    "        if char == '#':\n",
    "            break\n",
    "    cols[idc] = c[:idx-1]\n",
    "cols"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Assembly data columns: 44\n",
      "Initial inspection data columns: 16\n",
      "Final inspection data columns: 16\n"
     ]
    }
   ],
   "source": [
    "def redefine_cols(cols):\n",
    "    cols_asm = [c for c in cols if 'asm_' == c[:4]]\n",
    "    cols_int = [c for c in cols if 'int_' == c[:4]]\n",
    "    cols_fin = [c for c in cols if 'fin_' == c[:4]]\n",
    "    return cols_asm, cols_int, cols_fin\n",
    "cols_asm, cols_int, cols_fin = redefine_cols(cols)\n",
    "print(\"Assembly data columns: {}\".format(len(cols_asm)))\n",
    "print(\"Initial inspection data columns: {}\".format(len(cols_int)))\n",
    "print(\"Final inspection data columns: {}\".format(len(cols_fin)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.columns = cols"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exploratory Data Analysis\n",
    "This part involves EDA process. The first step is to find Nan."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Ti                        0.000000\n",
       "Da                        0.000000\n",
       "asm_Force                 0.000000\n",
       "asm_Length                0.000000\n",
       "asm_Height                0.000000\n",
       "                            ...   \n",
       "fin_Tracking, Zone4, G    0.042976\n",
       "fin_Shape                 0.042976\n",
       "fin_pressure              0.042976\n",
       "fin_flow                  0.042976\n",
       "fin_temperature           0.042976\n",
       "Length: 78, dtype: float64"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Find Nan\n",
    "df.isnull().mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are around 5% of Nan in this data set. From the readme file, this is due to quality stuffs or whatever. The Nan are only in the final inspection."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 16963 entries, 0 to 16962\n",
      "Data columns (total 78 columns):\n",
      "Ti                        16963 non-null object\n",
      "Da                        16963 non-null object\n",
      "asm_Force                 16963 non-null float64\n",
      "asm_Length                16963 non-null float64\n",
      "asm_Height                16963 non-null float64\n",
      "asm_Press-fit             16963 non-null float64\n",
      "asm_force                 16963 non-null float64\n",
      "asm_Length                16963 non-null float64\n",
      "asm_Depth                 16963 non-null float64\n",
      "asm_depth                 16963 non-null float64\n",
      "asm_Depth                 16963 non-null float64\n",
      "asm_parallelism           16963 non-null float64\n",
      "asm_force                 16963 non-null float64\n",
      "asm_Length                16963 non-null float64\n",
      "asm_force                 16963 non-null float64\n",
      "asm_Length                16963 non-null float64\n",
      "asm_torque 1              16963 non-null float64\n",
      "asm_angle 1               16963 non-null float64\n",
      "asm_torque 2              16963 non-null float64\n",
      "asm_angle 2               16963 non-null float64\n",
      "asm_torque 3              16963 non-null float64\n",
      "asm_angle 3               16963 non-null float64\n",
      "asm_torque 1              16963 non-null float64\n",
      "asm_torque 2              16963 non-null float64\n",
      "asm_torque 3              16963 non-null float64\n",
      "asm_angle 1               16963 non-null float64\n",
      "asm_angle 2               16963 non-null float64\n",
      "asm_angle 3               16963 non-null float64\n",
      "asm_torque 1              16963 non-null float64\n",
      "asm_torque 2              16963 non-null float64\n",
      "asm_torque 3              16963 non-null float64\n",
      "asm_torque 4              16963 non-null float64\n",
      "asm_angle 1               16963 non-null float64\n",
      "asm_angle 2               16963 non-null float64\n",
      "asm_angle 3               16963 non-null float64\n",
      "asm_angle 4               16963 non-null float64\n",
      "asm_temperature           16963 non-null float64\n",
      "asm_force                 16963 non-null float64\n",
      "asm_Length                16963 non-null float64\n",
      "asm_force                 16963 non-null float64\n",
      "asm_Temperature           16963 non-null float64\n",
      "asm_angle                 16963 non-null float64\n",
      "asm_torque                16963 non-null float64\n",
      "asm_angle                 16963 non-null float64\n",
      "asm_torque                16963 non-null float64\n",
      "asm_clearance             16963 non-null float64\n",
      "int_weight                16963 non-null float64\n",
      "int_Angle position        16963 non-null float64\n",
      "int_weight                16963 non-null float64\n",
      "int_Angle position        16963 non-null float64\n",
      "int_Tracking, Zone1       16963 non-null int64\n",
      "int_Tracking, Zone2       16963 non-null int64\n",
      "int_Tracking, Zone3       16963 non-null int64\n",
      "int_Tracking, Zone4       16963 non-null int64\n",
      "int_Tracking, Zone1, G    16963 non-null float64\n",
      "int_Tracking, Zone2, G    16963 non-null float64\n",
      "int_Tracking, Zone3, G    16963 non-null float64\n",
      "int_Tracking, Zone4, G    16963 non-null float64\n",
      "int_Shape                 16963 non-null object\n",
      "int_pressure              16963 non-null float64\n",
      "int_flow                  16963 non-null float64\n",
      "int_temperature           16963 non-null float64\n",
      "fin_weight                16234 non-null float64\n",
      "fin_Angle                 16234 non-null float64\n",
      "fin_weight                16234 non-null float64\n",
      "fin_Angle                 16234 non-null float64\n",
      "fin_Tracking, Zoe1        16234 non-null float64\n",
      "fin_Tracking, Zone2       16234 non-null float64\n",
      "fin_Tracking, Zone3       16234 non-null float64\n",
      "fin_Tracking, Zone4       16234 non-null float64\n",
      "fin_Tracking, Zone1, G    16234 non-null float64\n",
      "fin_Tracking, Zone2, G    16234 non-null float64\n",
      "fin_Tracking, Zone3, G    16234 non-null float64\n",
      "fin_Tracking, Zone4, G    16234 non-null float64\n",
      "fin_Shape                 16234 non-null object\n",
      "fin_pressure              16234 non-null float64\n",
      "fin_flow                  16234 non-null float64\n",
      "fin_temperature           16234 non-null float64\n",
      "dtypes: float64(70), int64(4), object(4)\n",
      "memory usage: 10.1+ MB\n"
     ]
    }
   ],
   "source": [
    "# Check data type\n",
    "df.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Most of the data are number variables. There are `Da`, `Ti`, `int_Shape`, and `fin_Shape` which are categorical variables. These variables shall be encoded afterwards."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "# After this point will be data processing. The initial dataframe shall be backed up first.\n",
    "df_bkup = df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Nan processing\n",
    "This section dedicates for Nan processing. The quick-dirty process is to remove the Nan out. If there is time left, data imputation can be done."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Ti                        0.0\n",
       "Da                        0.0\n",
       "asm_Force                 0.0\n",
       "asm_Length                0.0\n",
       "asm_Height                0.0\n",
       "                         ... \n",
       "fin_Tracking, Zone4, G    0.0\n",
       "fin_Shape                 0.0\n",
       "fin_pressure              0.0\n",
       "fin_flow                  0.0\n",
       "fin_temperature           0.0\n",
       "Length: 78, dtype: float64"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# remove nan\n",
    "df = df.dropna()\n",
    "df.isnull().mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Train-test split\n",
    "Do `train_test_split` here to avoid information leakage. But first of all the label columns should be identified."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/peerapon/opt/anaconda3/lib/python3.7/site-packages/pandas/core/frame.py:4102: SettingWithCopyWarning: \n",
      "A value is trying to be set on a copy of a slice from a DataFrame\n",
      "\n",
      "See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
      "  errors=errors,\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "((16234, 76), (11364, 76), (4870, 76))"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# This train_test_split is done under an assumption that the time sequence does not matter.\n",
    "# There will be another analysis that consider the sequence of data. \n",
    "\n",
    "\n",
    "# X_train, X_test, y_train, y_test = train_test_split(df.drop(columns=['Da', 'Ti']+label_cols), \n",
    "#                                                     df[label_cols], \n",
    "#                                                     test_size=0.3, shuffle=False)\n",
    "\n",
    "# This split considers sequence. \n",
    "df.drop(columns=['Da','Ti'], inplace=True)\n",
    "nrows = df.shape[0]\n",
    "test_size = int(0.3*nrows)\n",
    "\n",
    "train = df.head(nrows-test_size)\n",
    "test = df.tail(test_size)\n",
    "\n",
    "df.shape, train.shape, test.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Transform categorical data to numerical"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['asm_Force',\n",
       " 'asm_Length',\n",
       " 'asm_Height',\n",
       " 'asm_Press-fit',\n",
       " 'asm_force',\n",
       " 'asm_Length',\n",
       " 'asm_Depth',\n",
       " 'asm_depth',\n",
       " 'asm_Depth',\n",
       " 'asm_parallelism',\n",
       " 'asm_force',\n",
       " 'asm_Length',\n",
       " 'asm_force',\n",
       " 'asm_Length',\n",
       " 'asm_torque 1',\n",
       " 'asm_angle 1',\n",
       " 'asm_torque 2',\n",
       " 'asm_angle 2',\n",
       " 'asm_torque 3',\n",
       " 'asm_angle 3',\n",
       " 'asm_torque 1',\n",
       " 'asm_torque 2',\n",
       " 'asm_torque 3',\n",
       " 'asm_angle 1',\n",
       " 'asm_angle 2',\n",
       " 'asm_angle 3',\n",
       " 'asm_torque 1',\n",
       " 'asm_torque 2',\n",
       " 'asm_torque 3',\n",
       " 'asm_torque 4',\n",
       " 'asm_angle 1',\n",
       " 'asm_angle 2',\n",
       " 'asm_angle 3',\n",
       " 'asm_angle 4',\n",
       " 'asm_temperature',\n",
       " 'asm_force',\n",
       " 'asm_Length',\n",
       " 'asm_force',\n",
       " 'asm_Temperature',\n",
       " 'asm_angle',\n",
       " 'asm_torque',\n",
       " 'asm_angle',\n",
       " 'asm_torque',\n",
       " 'asm_clearance',\n",
       " 'int_weight',\n",
       " 'int_Angle position ',\n",
       " 'int_weight',\n",
       " 'int_Angle position',\n",
       " 'int_Tracking, Zone1',\n",
       " 'int_Tracking, Zone2',\n",
       " 'int_Tracking, Zone3',\n",
       " 'int_Tracking, Zone4',\n",
       " 'int_Tracking, Zone1, G',\n",
       " 'int_Tracking, Zone2, G',\n",
       " 'int_Tracking, Zone3, G',\n",
       " 'int_Tracking, Zone4, G',\n",
       " 'int_pressure',\n",
       " 'int_flow',\n",
       " 'int_temperature',\n",
       " 'fin_weight',\n",
       " 'fin_Angle',\n",
       " 'fin_weight',\n",
       " 'fin_Angle',\n",
       " 'fin_Tracking, Zoe1',\n",
       " 'fin_Tracking, Zone2',\n",
       " 'fin_Tracking, Zone3',\n",
       " 'fin_Tracking, Zone4',\n",
       " 'fin_Tracking, Zone1, G',\n",
       " 'fin_Tracking, Zone2, G',\n",
       " 'fin_Tracking, Zone3, G',\n",
       " 'fin_Tracking, Zone4, G',\n",
       " 'fin_pressure',\n",
       " 'fin_flow',\n",
       " 'fin_temperature']"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# distinguish between categorical and numerical var\n",
    "catcols = ['int_Shape', 'fin_Shape']\n",
    "numcols = [c for c in cols[2:] if c not in catcols]\n",
    "numcols"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we have lists that store categorical column names and numerical column names."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array(['Ascending', 'V-shape', 'Descending', 'Flat'], dtype=object),\n",
       " array(['Ascending', 'V-shape', 'Descending', 'Flat'], dtype=object))"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# check if the categorical column of the training data and testing data match each other.\n",
    "train[catcols[0]].unique(), train[catcols[0]].unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array(['V-shape', 'Ascending', 'Descending', 'Flat'], dtype=object),\n",
       " array(['V-shape', 'Ascending', 'Descending', 'Flat'], dtype=object))"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train[catcols[1]].unique(), train[catcols[1]].unique()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The two categorical variables have identical categories. This means that there is no missing category in both sets. \n",
    "\n",
    "There could be the case that the training set does not cover all categories of the testing set (underfitting) as well as the case that the training set covers more categories than the testing set (overfitting). Therefore, please do not forget to check it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>asm_Force</th>\n",
       "      <th>asm_Length</th>\n",
       "      <th>asm_Height</th>\n",
       "      <th>asm_Press-fit</th>\n",
       "      <th>asm_force</th>\n",
       "      <th>asm_Length</th>\n",
       "      <th>asm_Depth</th>\n",
       "      <th>asm_depth</th>\n",
       "      <th>asm_Depth</th>\n",
       "      <th>asm_parallelism</th>\n",
       "      <th>...</th>\n",
       "      <th>fin_flow</th>\n",
       "      <th>fin_temperature</th>\n",
       "      <th>int_Shape_Ascending</th>\n",
       "      <th>int_Shape_V-shape</th>\n",
       "      <th>int_Shape_Descending</th>\n",
       "      <th>int_Shape_Flat</th>\n",
       "      <th>fin_Shape_V-shape</th>\n",
       "      <th>fin_Shape_Ascending</th>\n",
       "      <th>fin_Shape_Descending</th>\n",
       "      <th>fin_Shape_Flat</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>0.149</td>\n",
       "      <td>102.52</td>\n",
       "      <td>0.215</td>\n",
       "      <td>12.747</td>\n",
       "      <td>1.094</td>\n",
       "      <td>135.43</td>\n",
       "      <td>1.8541</td>\n",
       "      <td>1.8517</td>\n",
       "      <td>1.8535</td>\n",
       "      <td>0.0024</td>\n",
       "      <td>...</td>\n",
       "      <td>0.828</td>\n",
       "      <td>88.15</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>0.163</td>\n",
       "      <td>102.61</td>\n",
       "      <td>0.191</td>\n",
       "      <td>12.731</td>\n",
       "      <td>1.116</td>\n",
       "      <td>135.48</td>\n",
       "      <td>1.8583</td>\n",
       "      <td>1.8556</td>\n",
       "      <td>1.8602</td>\n",
       "      <td>0.0046</td>\n",
       "      <td>...</td>\n",
       "      <td>1.068</td>\n",
       "      <td>89.14</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>0.160</td>\n",
       "      <td>102.56</td>\n",
       "      <td>0.192</td>\n",
       "      <td>12.734</td>\n",
       "      <td>1.033</td>\n",
       "      <td>135.48</td>\n",
       "      <td>1.8533</td>\n",
       "      <td>1.8487</td>\n",
       "      <td>1.8526</td>\n",
       "      <td>0.0046</td>\n",
       "      <td>...</td>\n",
       "      <td>0.828</td>\n",
       "      <td>87.30</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>0.158</td>\n",
       "      <td>102.47</td>\n",
       "      <td>0.195</td>\n",
       "      <td>12.726</td>\n",
       "      <td>1.079</td>\n",
       "      <td>135.48</td>\n",
       "      <td>1.8578</td>\n",
       "      <td>1.8546</td>\n",
       "      <td>1.8583</td>\n",
       "      <td>0.0037</td>\n",
       "      <td>...</td>\n",
       "      <td>1.068</td>\n",
       "      <td>88.68</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>0.154</td>\n",
       "      <td>102.61</td>\n",
       "      <td>0.207</td>\n",
       "      <td>12.752</td>\n",
       "      <td>1.061</td>\n",
       "      <td>135.48</td>\n",
       "      <td>1.8594</td>\n",
       "      <td>1.8543</td>\n",
       "      <td>1.8616</td>\n",
       "      <td>0.0073</td>\n",
       "      <td>...</td>\n",
       "      <td>0.816</td>\n",
       "      <td>87.29</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 82 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   asm_Force  asm_Length  asm_Height  asm_Press-fit  asm_force  asm_Length  \\\n",
       "0      0.149      102.52       0.215         12.747      1.094      135.43   \n",
       "1      0.163      102.61       0.191         12.731      1.116      135.48   \n",
       "2      0.160      102.56       0.192         12.734      1.033      135.48   \n",
       "3      0.158      102.47       0.195         12.726      1.079      135.48   \n",
       "4      0.154      102.61       0.207         12.752      1.061      135.48   \n",
       "\n",
       "   asm_Depth  asm_depth  asm_Depth  asm_parallelism  ...  fin_flow  \\\n",
       "0     1.8541     1.8517     1.8535           0.0024  ...     0.828   \n",
       "1     1.8583     1.8556     1.8602           0.0046  ...     1.068   \n",
       "2     1.8533     1.8487     1.8526           0.0046  ...     0.828   \n",
       "3     1.8578     1.8546     1.8583           0.0037  ...     1.068   \n",
       "4     1.8594     1.8543     1.8616           0.0073  ...     0.816   \n",
       "\n",
       "   fin_temperature  int_Shape_Ascending  int_Shape_V-shape  \\\n",
       "0            88.15                    1                  0   \n",
       "1            89.14                    0                  1   \n",
       "2            87.30                    0                  1   \n",
       "3            88.68                    0                  1   \n",
       "4            87.29                    0                  0   \n",
       "\n",
       "   int_Shape_Descending  int_Shape_Flat  fin_Shape_V-shape  \\\n",
       "0                     0               0                  1   \n",
       "1                     0               0                  1   \n",
       "2                     0               0                  1   \n",
       "3                     0               0                  0   \n",
       "4                     1               0                  0   \n",
       "\n",
       "   fin_Shape_Ascending  fin_Shape_Descending  fin_Shape_Flat  \n",
       "0                    0                     0               0  \n",
       "1                    0                     0               0  \n",
       "2                    0                     0               0  \n",
       "3                    1                     0               0  \n",
       "4                    1                     0               0  \n",
       "\n",
       "[5 rows x 82 columns]"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# One-hot encoder here is used here based on `feature_engine` library instead of `sklearn`. \n",
    "# This library returns transformed data as dataframe instead of numpy matrix as sklearn.\n",
    "ohe_enc = OneHotCategoricalEncoder(\n",
    "    top_categories=None,\n",
    "    variables = catcols,\n",
    "    drop_last=False)\n",
    "ohe_enc.fit(train)\n",
    "train = ohe_enc.transform(train)\n",
    "test = ohe_enc.transform(test)\n",
    "train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of initial inspection columns atfter OneHotEncoding:  19\n",
      "Number of final inspection columns atfter OneHotEncoding:  19\n"
     ]
    }
   ],
   "source": [
    "# After modifying the columns, redefine the column names\n",
    "cols_asm, cols_int, cols_fin = redefine_cols(train.columns)\n",
    "print(\"Number of initial inspection columns atfter OneHotEncoding: \", len(cols_int))\n",
    "print(\"Number of final inspection columns atfter OneHotEncoding: \", len(cols_fin))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Scale the numerical data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>int_weight</th>\n",
       "      <th>int_weight</th>\n",
       "      <th>int_Angle position</th>\n",
       "      <th>int_weight</th>\n",
       "      <th>int_weight</th>\n",
       "      <th>int_Angle position</th>\n",
       "      <th>int_Tracking, Zone1</th>\n",
       "      <th>int_Tracking, Zone2</th>\n",
       "      <th>int_Tracking, Zone3</th>\n",
       "      <th>int_Tracking, Zone4</th>\n",
       "      <th>...</th>\n",
       "      <th>fin_Tracking, Zone2, G</th>\n",
       "      <th>fin_Tracking, Zone3, G</th>\n",
       "      <th>fin_Tracking, Zone4, G</th>\n",
       "      <th>fin_pressure</th>\n",
       "      <th>fin_flow</th>\n",
       "      <th>fin_temperature</th>\n",
       "      <th>fin_Shape_V-shape</th>\n",
       "      <th>fin_Shape_Ascending</th>\n",
       "      <th>fin_Shape_Descending</th>\n",
       "      <th>fin_Shape_Flat</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>count</td>\n",
       "      <td>11364.0</td>\n",
       "      <td>11364.0</td>\n",
       "      <td>11364.0</td>\n",
       "      <td>11364.0</td>\n",
       "      <td>11364.0</td>\n",
       "      <td>11364.0</td>\n",
       "      <td>11364.0</td>\n",
       "      <td>11364.0</td>\n",
       "      <td>11364.0</td>\n",
       "      <td>11364.0</td>\n",
       "      <td>...</td>\n",
       "      <td>11364.0</td>\n",
       "      <td>11364.0</td>\n",
       "      <td>11364.0</td>\n",
       "      <td>11364.0</td>\n",
       "      <td>11364.0</td>\n",
       "      <td>11364.0</td>\n",
       "      <td>11364.0</td>\n",
       "      <td>11364.0</td>\n",
       "      <td>11364.0</td>\n",
       "      <td>11364.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>mean</td>\n",
       "      <td>7.5</td>\n",
       "      <td>26.4</td>\n",
       "      <td>181.0</td>\n",
       "      <td>7.5</td>\n",
       "      <td>26.4</td>\n",
       "      <td>189.5</td>\n",
       "      <td>56942.9</td>\n",
       "      <td>72063.9</td>\n",
       "      <td>113281.7</td>\n",
       "      <td>134252.3</td>\n",
       "      <td>...</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0.2</td>\n",
       "      <td>2.9</td>\n",
       "      <td>1.0</td>\n",
       "      <td>88.4</td>\n",
       "      <td>0.4</td>\n",
       "      <td>0.5</td>\n",
       "      <td>0.1</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>std</td>\n",
       "      <td>3.9</td>\n",
       "      <td>13.9</td>\n",
       "      <td>102.4</td>\n",
       "      <td>3.9</td>\n",
       "      <td>13.9</td>\n",
       "      <td>115.9</td>\n",
       "      <td>3759.1</td>\n",
       "      <td>12081.7</td>\n",
       "      <td>9832.7</td>\n",
       "      <td>7074.8</td>\n",
       "      <td>...</td>\n",
       "      <td>0.1</td>\n",
       "      <td>0.1</td>\n",
       "      <td>0.1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.1</td>\n",
       "      <td>0.9</td>\n",
       "      <td>0.5</td>\n",
       "      <td>0.5</td>\n",
       "      <td>0.3</td>\n",
       "      <td>0.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>min</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.3</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.3</td>\n",
       "      <td>-109.5</td>\n",
       "      <td>40000.0</td>\n",
       "      <td>65000.0</td>\n",
       "      <td>96000.0</td>\n",
       "      <td>120000.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.8</td>\n",
       "      <td>0.1</td>\n",
       "      <td>82.3</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>25%</td>\n",
       "      <td>4.6</td>\n",
       "      <td>16.0</td>\n",
       "      <td>91.1</td>\n",
       "      <td>4.6</td>\n",
       "      <td>16.0</td>\n",
       "      <td>97.4</td>\n",
       "      <td>54700.0</td>\n",
       "      <td>65000.0</td>\n",
       "      <td>109400.0</td>\n",
       "      <td>129800.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.1</td>\n",
       "      <td>0.1</td>\n",
       "      <td>0.1</td>\n",
       "      <td>2.9</td>\n",
       "      <td>1.0</td>\n",
       "      <td>87.8</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>50%</td>\n",
       "      <td>7.1</td>\n",
       "      <td>24.3</td>\n",
       "      <td>180.4</td>\n",
       "      <td>7.1</td>\n",
       "      <td>24.3</td>\n",
       "      <td>190.8</td>\n",
       "      <td>55400.0</td>\n",
       "      <td>65700.0</td>\n",
       "      <td>119700.0</td>\n",
       "      <td>138000.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.1</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0.2</td>\n",
       "      <td>2.9</td>\n",
       "      <td>1.0</td>\n",
       "      <td>88.4</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>75%</td>\n",
       "      <td>9.9</td>\n",
       "      <td>34.7</td>\n",
       "      <td>270.8</td>\n",
       "      <td>9.9</td>\n",
       "      <td>34.7</td>\n",
       "      <td>282.4</td>\n",
       "      <td>56800.0</td>\n",
       "      <td>68200.0</td>\n",
       "      <td>120000.0</td>\n",
       "      <td>140000.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0.2</td>\n",
       "      <td>0.3</td>\n",
       "      <td>2.9</td>\n",
       "      <td>1.0</td>\n",
       "      <td>89.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>max</td>\n",
       "      <td>24.6</td>\n",
       "      <td>89.5</td>\n",
       "      <td>360.0</td>\n",
       "      <td>24.6</td>\n",
       "      <td>89.5</td>\n",
       "      <td>459.8</td>\n",
       "      <td>65000.0</td>\n",
       "      <td>96000.0</td>\n",
       "      <td>120000.0</td>\n",
       "      <td>140000.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.4</td>\n",
       "      <td>1.4</td>\n",
       "      <td>1.6</td>\n",
       "      <td>3.1</td>\n",
       "      <td>1.1</td>\n",
       "      <td>93.3</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>8 rows × 44 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       int_weight  int_weight  int_Angle position   int_weight  int_weight  \\\n",
       "count     11364.0     11364.0              11364.0     11364.0     11364.0   \n",
       "mean          7.5        26.4                181.0         7.5        26.4   \n",
       "std           3.9        13.9                102.4         3.9        13.9   \n",
       "min           0.0         0.3                  0.0         0.0         0.3   \n",
       "25%           4.6        16.0                 91.1         4.6        16.0   \n",
       "50%           7.1        24.3                180.4         7.1        24.3   \n",
       "75%           9.9        34.7                270.8         9.9        34.7   \n",
       "max          24.6        89.5                360.0        24.6        89.5   \n",
       "\n",
       "       int_Angle position  int_Tracking, Zone1  int_Tracking, Zone2  \\\n",
       "count             11364.0              11364.0              11364.0   \n",
       "mean                189.5              56942.9              72063.9   \n",
       "std                 115.9               3759.1              12081.7   \n",
       "min                -109.5              40000.0              65000.0   \n",
       "25%                  97.4              54700.0              65000.0   \n",
       "50%                 190.8              55400.0              65700.0   \n",
       "75%                 282.4              56800.0              68200.0   \n",
       "max                 459.8              65000.0              96000.0   \n",
       "\n",
       "       int_Tracking, Zone3  int_Tracking, Zone4  ...  fin_Tracking, Zone2, G  \\\n",
       "count              11364.0              11364.0  ...                 11364.0   \n",
       "mean              113281.7             134252.3  ...                     0.2   \n",
       "std                 9832.7               7074.8  ...                     0.1   \n",
       "min                96000.0             120000.0  ...                     0.0   \n",
       "25%               109400.0             129800.0  ...                     0.1   \n",
       "50%               119700.0             138000.0  ...                     0.1   \n",
       "75%               120000.0             140000.0  ...                     0.2   \n",
       "max               120000.0             140000.0  ...                     1.4   \n",
       "\n",
       "       fin_Tracking, Zone3, G  fin_Tracking, Zone4, G  fin_pressure  fin_flow  \\\n",
       "count                 11364.0                 11364.0       11364.0   11364.0   \n",
       "mean                      0.2                     0.2           2.9       1.0   \n",
       "std                       0.1                     0.1           0.0       0.1   \n",
       "min                       0.0                     0.0           2.8       0.1   \n",
       "25%                       0.1                     0.1           2.9       1.0   \n",
       "50%                       0.2                     0.2           2.9       1.0   \n",
       "75%                       0.2                     0.3           2.9       1.0   \n",
       "max                       1.4                     1.6           3.1       1.1   \n",
       "\n",
       "       fin_temperature  fin_Shape_V-shape  fin_Shape_Ascending  \\\n",
       "count          11364.0            11364.0              11364.0   \n",
       "mean              88.4                0.4                  0.5   \n",
       "std                0.9                0.5                  0.5   \n",
       "min               82.3                0.0                  0.0   \n",
       "25%               87.8                0.0                  0.0   \n",
       "50%               88.4                0.0                  0.0   \n",
       "75%               89.0                1.0                  1.0   \n",
       "max               93.3                1.0                  1.0   \n",
       "\n",
       "       fin_Shape_Descending  fin_Shape_Flat  \n",
       "count               11364.0         11364.0  \n",
       "mean                    0.1             0.0  \n",
       "std                     0.3             0.2  \n",
       "min                     0.0             0.0  \n",
       "25%                     0.0             0.0  \n",
       "50%                     0.0             0.0  \n",
       "75%                     0.0             0.0  \n",
       "max                     1.0             1.0  \n",
       "\n",
       "[8 rows x 44 columns]"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# let's first have a look on statistical information\n",
    "np.round(train[cols_int+cols_fin].describe(), 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/peerapon/opt/anaconda3/lib/python3.7/site-packages/sklearn/preprocessing/data.py:323: DataConversionWarning: Data with input dtype int64, float64 were all converted to float64 by MinMaxScaler.\n",
      "  return self.partial_fit(X, y)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>asm_Force</th>\n",
       "      <th>asm_Length</th>\n",
       "      <th>asm_Height</th>\n",
       "      <th>asm_Press-fit</th>\n",
       "      <th>asm_force</th>\n",
       "      <th>asm_Length</th>\n",
       "      <th>asm_Depth</th>\n",
       "      <th>asm_depth</th>\n",
       "      <th>asm_Depth</th>\n",
       "      <th>asm_parallelism</th>\n",
       "      <th>...</th>\n",
       "      <th>fin_flow</th>\n",
       "      <th>fin_temperature</th>\n",
       "      <th>int_Shape_Ascending</th>\n",
       "      <th>int_Shape_V-shape</th>\n",
       "      <th>int_Shape_Descending</th>\n",
       "      <th>int_Shape_Flat</th>\n",
       "      <th>fin_Shape_V-shape</th>\n",
       "      <th>fin_Shape_Ascending</th>\n",
       "      <th>fin_Shape_Descending</th>\n",
       "      <th>fin_Shape_Flat</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>0.347826</td>\n",
       "      <td>0.997179</td>\n",
       "      <td>0.523248</td>\n",
       "      <td>0.807539</td>\n",
       "      <td>0.555048</td>\n",
       "      <td>0.994274</td>\n",
       "      <td>0.595506</td>\n",
       "      <td>0.560773</td>\n",
       "      <td>0.991176</td>\n",
       "      <td>0.110599</td>\n",
       "      <td>...</td>\n",
       "      <td>0.698795</td>\n",
       "      <td>0.529626</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>0.469565</td>\n",
       "      <td>0.998055</td>\n",
       "      <td>0.519205</td>\n",
       "      <td>0.806525</td>\n",
       "      <td>0.566210</td>\n",
       "      <td>0.994641</td>\n",
       "      <td>0.713483</td>\n",
       "      <td>0.668508</td>\n",
       "      <td>0.994759</td>\n",
       "      <td>0.211982</td>\n",
       "      <td>...</td>\n",
       "      <td>0.939759</td>\n",
       "      <td>0.619872</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>0.443478</td>\n",
       "      <td>0.997568</td>\n",
       "      <td>0.519373</td>\n",
       "      <td>0.806715</td>\n",
       "      <td>0.524099</td>\n",
       "      <td>0.994641</td>\n",
       "      <td>0.573034</td>\n",
       "      <td>0.477901</td>\n",
       "      <td>0.990695</td>\n",
       "      <td>0.211982</td>\n",
       "      <td>...</td>\n",
       "      <td>0.698795</td>\n",
       "      <td>0.452142</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>0.426087</td>\n",
       "      <td>0.996693</td>\n",
       "      <td>0.519879</td>\n",
       "      <td>0.806208</td>\n",
       "      <td>0.547438</td>\n",
       "      <td>0.994641</td>\n",
       "      <td>0.699438</td>\n",
       "      <td>0.640884</td>\n",
       "      <td>0.993743</td>\n",
       "      <td>0.170507</td>\n",
       "      <td>...</td>\n",
       "      <td>0.939759</td>\n",
       "      <td>0.577940</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>0.391304</td>\n",
       "      <td>0.998055</td>\n",
       "      <td>0.521900</td>\n",
       "      <td>0.807856</td>\n",
       "      <td>0.538305</td>\n",
       "      <td>0.994641</td>\n",
       "      <td>0.744382</td>\n",
       "      <td>0.632597</td>\n",
       "      <td>0.995508</td>\n",
       "      <td>0.336406</td>\n",
       "      <td>...</td>\n",
       "      <td>0.686747</td>\n",
       "      <td>0.451231</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 82 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   asm_Force  asm_Length  asm_Height  asm_Press-fit  asm_force  asm_Length  \\\n",
       "0   0.347826    0.997179    0.523248       0.807539   0.555048    0.994274   \n",
       "1   0.469565    0.998055    0.519205       0.806525   0.566210    0.994641   \n",
       "2   0.443478    0.997568    0.519373       0.806715   0.524099    0.994641   \n",
       "3   0.426087    0.996693    0.519879       0.806208   0.547438    0.994641   \n",
       "4   0.391304    0.998055    0.521900       0.807856   0.538305    0.994641   \n",
       "\n",
       "   asm_Depth  asm_depth  asm_Depth  asm_parallelism  ...  fin_flow  \\\n",
       "0   0.595506   0.560773   0.991176         0.110599  ...  0.698795   \n",
       "1   0.713483   0.668508   0.994759         0.211982  ...  0.939759   \n",
       "2   0.573034   0.477901   0.990695         0.211982  ...  0.698795   \n",
       "3   0.699438   0.640884   0.993743         0.170507  ...  0.939759   \n",
       "4   0.744382   0.632597   0.995508         0.336406  ...  0.686747   \n",
       "\n",
       "   fin_temperature  int_Shape_Ascending  int_Shape_V-shape  \\\n",
       "0         0.529626                  1.0                0.0   \n",
       "1         0.619872                  0.0                1.0   \n",
       "2         0.452142                  0.0                1.0   \n",
       "3         0.577940                  0.0                1.0   \n",
       "4         0.451231                  0.0                0.0   \n",
       "\n",
       "   int_Shape_Descending  int_Shape_Flat  fin_Shape_V-shape  \\\n",
       "0                   0.0             0.0                1.0   \n",
       "1                   0.0             0.0                1.0   \n",
       "2                   0.0             0.0                1.0   \n",
       "3                   0.0             0.0                0.0   \n",
       "4                   1.0             0.0                0.0   \n",
       "\n",
       "   fin_Shape_Ascending  fin_Shape_Descending  fin_Shape_Flat  \n",
       "0                  0.0                   0.0             0.0  \n",
       "1                  0.0                   0.0             0.0  \n",
       "2                  0.0                   0.0             0.0  \n",
       "3                  1.0                   0.0             0.0  \n",
       "4                  1.0                   0.0             0.0  \n",
       "\n",
       "[5 rows x 82 columns]"
      ]
     },
     "execution_count": 79,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create a scaler\n",
    "# scaler = StandardScaler()\n",
    "scaler = MinMaxScaler()\n",
    "scaler.fit(train)\n",
    "train_scaled_val = scaler.transform(train)\n",
    "test_scaled_val = scaler.transform(test)\n",
    "train_scaled = pd.DataFrame(train_scaled_val, index=train.index, columns=train.columns)\n",
    "test_scaled = pd.DataFrame(test_scaled_val, index=test.index, columns=test.columns)\n",
    "train_scaled.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Identify target label's tolerance from training set"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The problem statement here is to predict if the final temperature `fin_temperature` is still inside the tolerance of +- 2 SD or not. Therefore, the problem becomes classification problem instead of regression problem.\n",
    "\n",
    "This tolerance cut-off can be done only if the data is normally distributed. It is found from another analysis that not every column distribute this way. Some are skewed, some are bimodal. The `fin_temperature` itself is also not beautifully normally distributed but let's assume the cut-off works."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXgAAAE/CAYAAABMwNhyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3deXwkd3nn8c8jjUYaXXNqPIcP2WQMGFgfGQjBIYBDjLExTshmY4LDEYhDAuEICQwQEoeFxUteC4QlG3CcYGIHDJjTBwGfQDC2GYPHBz7n8pwazYw0uu9n/6hqTY+sozWqX1V31/f9eunVre7qrqerpa9+eurXVebuiIhI9anJugAREQlDAS8iUqUU8CIiVUoBLyJSpRTwIiJVSgEvIlKlFPCSCDNzM+s3s4/PcP+zzewXZtZrZu8ys8+b2UfSrlMiZna6mfWZ2biZvS3reiSMRVkXIFXlTHd/CsDM2oG73L09vu/98fdnJ7UyM7sG2O3uf5PUc2bFzF4OXOfuJwZcxxUA7n6Fuz8BNJvZXaHWJ9nTCF7ScgrwSNZFZMEiQX/XzEyDNXkGBbwEZ2Z3AK8APhe3BU43s2vM7GPx/S83s91m9j4zO2Bm+8zsLXM85+XAG4D3x895Y3z7OjP7hpl1mtl2M3tX0WOuMLOvm9l1cavoobiWD8br3WVm5xctf5eZfcLM7jOzI2b2HTNbUXT/i83sbjPrNrMt8Si8+LEfN7OfAAPAaWb2FjN7NF73NjP703jZJuB7wLr4tfTFr2NyGxVvp6Lvd5jZB8zsQaDfzBbN9volfxTwEoS77yi0Z9z9PODHwDvdvTluD0y1BlgKrAfeCvyTmS2f5fmvAv4D+GT8nBfHo+QbgS3x8/wW8B4ze1XRQy8GrgWWA78Avk/0e7Ae+CjwhSmreiPwx8A6YAz4LICZrQduBj4GrAD+CviGmbUVPfaPgMuBFmAncAB4DdAKvAX4tJmd4+79wKuBvfFraXb3vTO99ileD1wELAMmZnv9cWvmihKfV6qAAl7KxSjwUXcfdfdbgD7g2fN8jhcCbe7+UXcfcfdtwL8AlxYt82N3/767jwFfB9qAK919FLgeaDezZUXLX+vuD8ch/BHgf5hZLXAZcIu73+LuE+5+K7AZuLDosde4+yPuPha/rpvdfatHfgj8AHjpPF/jVJ91913uPlji65ccUd9OysWhOHQLBoDmeT7HKURtju6i22qJ/nso6Ci6PggcdPfxou+J11t4jl1Fy+8E6oBV8bp+38wuLrq/Driz6Pvix2Jmrwb+DjidaHDVCDxU0iubWfE6Snn9kiMKeKlkUw+FugvY7u4bElzHSUXXTyb6T+NgvK5r3f1PSqnPzOqBbxC1fL7j7qNm9m3Api5bpJ/oj0DBmtnWQZjXLxVMLRqpZB3AaUXf3wf0xDsel5hZrZk938xeuIB1XGZmZ5hZI1GP/oZ4xH8dcLGZvSpeT0O8E3SmaY6LgXqgExiLR/PnF93fAaw0s6VFtz0AXGhmK8xsDfCeOWoN8fqlgingpZL9K3BGPIvl23HwXgycBWwnGmlfTbTz9nhdC1wD7AcagHcBuPsu4BLgQ0ShvQv4a2b4nXL33vixXwO6gD8Evlt0/2PAV4Bt8etZF697C7CDqF//1dkKDfT6pYKZTvghSTCzIWCYaKdfVXxCNf4Q0HXufnXWtSTNzDYAPyP6z+LP3f2abCuSENSDl0S4e0PWNUjp3P1JoqmVUsXUopGyZmaPFH34p/jrDVnXJlLu1KIREalSGsGLiFQpBbyISJUqq52sq1at8vb29qzLEBGpGPfff/9Bd2+b7r6yCvj29nY2b96cdRkiIhXDzHbOdJ9aNCIiVUoBLyJSpRTwIiJVSgEvIlKlFPAiIlVKAS8iUqUU8CIiVUoBLyJSpRTwIiJVSgEvIlKlFPAiIlWqrI5FI1It2jfdPHl9x5UXZViJ5JlG8CIiVUoBLyJSpRTwIiJVSgEvIlKlFPAiIlVKAS8iUqWCTpM0sx1ALzAOjLn7xpDrExGRo9KYB/8Kdz+YwnpERKSIWjQiIlUqdMA78AMzu9/MLg+8LhERKRK6RXOuu+81s9XArWb2mLv/qHiBOPgvBzj55JMDlyMikh9BR/Duvje+PAB8C3jRNMtc5e4b3X1jW1tbyHJERHIlWMCbWZOZtRSuA+cDD4dan4iIHCtki+YE4FtmVljPl939PwOuT0REigQLeHffBpwZ6vlFRGR2miYpIlKlFPAiIlVKAS8iUqUU8CJBOPWMZF2E5JwCXiSAz9V9lnvq38k6dBgmyY4CXiRpPXt5Te29LLc+Xlf746yrkRxTwIskbffPJq9urHkiw0Ik7xTwIknrjEL9xvEX87ya7eCecUGSVwp4kaT17uOwN3P/xOm0WQ/07s+6IskpBbxI0nr30+HLedxPir4/qDaNZEMBL5K03r10+Aqenlgdfd+1I9NyJL8U8CJJi0fw+1jJqNcq4CUzCniRJI2PQV8HHSxjghp2+yoFvGRGAS+SpP5O8Ak6fAUAu3y1Al4yo4AXSVLvXgA6fDkA+32FZtFIZhTwIkmKw3x/HPAdLIe+DpgYz7IqyanQJ90WqQrtm26evL7jyotmXrB3H3B0BH/Al4GPQ/9BaDkhaI0iU2kEL5Kknn1gtRxiKQAH4qCnT20aSZ8CXiRJvfuh+QQm4l+twkhefXjJggJeJEm9e6FlzeS3B3xZfPu+jAqSPFPAiySpdz+0rpv8tpNCwHdkVJDkmQJeJEm9+44ZwY+yCBpXagQvmVDAiyRldBAGu6Bl7bG3t6xVD14yoYAXSUohxKcGfNMqGNCp+yR9CniR+ZoYh+vfAJ86A/Y/fPT2QhumdUrAN66EgUPp1ScSU8CLzNeW6+Gxm6BnD9z1icmb3/mFWwD47aufPHZ5BbxkRJ9kFZmHekbgzv8F638V1p0ND3w56r3XLeEE6wKgozA1sqBxJQwdiY40WatfOUmPftpE5rLrPv5P3T9zwJexzg5Bz2743c/D+Aj87GrY8RPY8EpOsC4GfTE9NB37+MaV0eVgFzS3pV+/5JYCXmQG7Ztu5gW2jRsWX8H5NXU0MEKdjcNL3gWnvhSG+8BqYde9sOGVrLdO9vpKwI59osbo0MEMHFLAS6oU8CIzcv5n3RfpooULhq9klEW0WTd3nf+26O76Zljz/CjggZOsMzr++1SFEbz68JIy7WQVmcELbDtn1Wzlc2O/Qzct9LOEHT5lhsxJvwZ77ofxMU60zugMTlMp4CUjCniRGVxUew8jXst3xs+deaETXwQjfbD7PlZYn0bwUlbUohGZwW/UPMzP/XR6aTzm9mOODf+BF0VXtlwPwC6fpse+pKgHL5IijeBFpjPYzRm2k7vHnzf7cstOhuY18PMvAfCYn/zMZeoaYHEzDBwOUKjIzBTwItPpeIQacx7002ZfzgzafwOALm9mu6+ZfrklKzSCl9Qp4EWm0xEdguCXE6fMulj7ppu5+P6zOOKN/PPYxfhMv1JLlkYfdhJJkXrwItPpeJhD3sIBls256EN+GmcNXzVzuAM0LFPAS+oU8CLTObSVrb6OZ3xoaQazhXv7ppv5Qt0gJ1sHz02oPJFSqEUjMp2uHdNPeTxOPd5Iqw0k9nwipVDAi0w1Ngw9e3l6IsGAp4lWFPCSLgW8yFRHdgM+/Zz249TjjbTYYHRESZGUBA94M6s1s1+Y2U2h1yWSiK7tAMm2aAoflhruSew5ReaSxgj+3cCjKaxHJBldOwF4OtEefHwIYc2kkRQFDXgzOxG4CLg65HpEEtW9E2rrS5oiWarJEbwCXlIUegT/GeD9wETg9Ygk58geaF07+7z2edIIXrIQLODN7DXAAXe/f47lLjezzWa2ubOzM1Q5IqXr64iOL5MgjeAlCyFH8OcCrzWzHcD1wHlmdt3Uhdz9Knff6O4b29p0thspA737oeWERJ+yxxXwkr5gAe/uH3T3E929HbgUuMPdLwu1PpHEBBnBq0Uj6dM8eJFiIwPRVMaER/B9NDDhpoCXVKUS8O5+l7u/Jo11iSxI3/7oMuERvFNDL0sU8JIqjeBFivV2RJcJj+AhnkmjgJcUKeBFigUawUM8k0afZJUUKeBFilzx5TsAOOczDyX+3BrBS9oU8CJFVls3o15LF82JP3cPjQp4SZUCXqTIauumk6WJfoq1oFcBLylTwIsUWU0XBzy5Y9AU6/UlMKQevKRHAS9SpM2O0Bkq4FkS7WR1D/L8IlMp4EWKLLdeDntrkOfu9UbAYaQvyPOLTKWAFylwZzm9dNES5On7WBJdGe4N8vwiUyngRQpG+qi3MQ578jNooDCCR314SY0CXqRg4BBAsBF87+Rp+zSCl3Qo4EUK4oA/7IEC3gstGk2VlHQo4EUKBg4D0BUq4FGLRtKlgBcpKIzgQ+1kde1klXQp4EUKQrdoJmfRaAQv6VDAixQMHGLMa462UhLWTwNgGsFLahTwIgUDh+miOchxaCA66Qf1LerBS2oU8CIFA4eC7WCdVN+iEbykRgEvUjDYRXeAwwQfo75V0yQlNQp4kYKhbo54U9h1NLRqBC+pUcCLFAwdiU7KEZJ68JIiBbxIweCR6LR6IdW3apqkpEYBLwIwMQHDPfQQOuC1k1XSo4AXgXhU7fR44BZNQ6taNJIaBbwIwFA3AEeCj+BbYWwQxkfDrkcEBbxIJD4ZdvARfH18tii1aSQFCngROBrwoUfwDYWAV5tGwlPAi8BkwAefB18ff1JWfXhJgQJeBGAw6sGn16JRwEt4CngRSK9FUxjBqwcvKVDAi0Ac8EYfDWHX07A0Xp9G8BKeAl4EommSDUuDHSp40uQIXgEv4SngRSAaURdmuISkHrykSAEvAlFPvD6FgF9UDzV16sFLKhTwIgAjvbA48LHgAcx0uAJJjQJeBGC4D+pTCHjQESUlNQp4EYCRvnRG8KAjSkpqFPAikO4IvmGpWjSSCgW8CMQj+MAn3C7QCF5SooAXcY8CPtUevE68LeGVFPBm9g0zu8jMSv6DYGYNZnafmW0xs0fM7O+Pv0yRcJ77wW+CT/CJ23ens0KN4CUlpQb2PwN/CDxpZlea2XNKeMwwcJ67nwmcBVxgZi8+zjpFgmlmCID+0IcpANo33cznfnqAsYEj0X8OIgGVFPDufpu7vwE4B9gB3Gpmd5vZW8ysbobHuLv3xd/WxV/6iZay02SDAPR7+IAH6PVGFtkEjA6msj7Jr/m0XFYCbwbeBvwC+EeiwL91lsfUmtkDwAHgVne/d0HVigTQlOIIHqCPJdEVzYWXwErtwX8T+DHQCFzs7q9196+6+18AM+6Zcvdxdz8LOBF4kZk9f5rnvtzMNpvZ5s7OzuN7FSILUGjRTAZvYL2FY85rqqQEVuoI/mp3P8PdP+Hu+wDMrB7A3TfO9WB37wbuAi6Y5r6r3H2ju29sa2srvXKRhKTeopkcwWtHq4RVasB/bJrbfjrbA8yszcyWxdeXAK8EHptfeSLhpT+CLwS8pkpKWItmu9PM1gDrgSVmdjZg8V2tRO2a2awFvmRmtUR/SL7m7jctsF6RxKU9gu8r/OpoBC+BzRrwwKuIdqyeCHyq6PZe4EOzPdDdHwTOXkhxImk4upM15RG8evAS2KwB7+5fIhqF/567fyOlmkRS1VwYwac0i0Y9eEnLXC2ay9z9OqDdzP5y6v3u/qlpHiZSUZoYYsDrmUjpyB1HWzQawUtYc7VoCqeYT+kgHSLpa2IwtdE7wAQ19HkDzWrRSGBztWi+EF/qODJStZptiL6UdrAW9LGEZo3gJbBSP+j0STNrNbM6M7vdzA6a2WWhixNJQxNDqe1gLej1RrVoJLhSm47nu3sP8BpgN3A68NfBqhJJUbOl26KBeM69drJKYKUGfOGAYhcCX3H3w4HqEUldE0P0edoj+CWaJinBlRrwN5rZY8BG4HYza4N48rBIhUt7JyvEUyU1gpfASj1c8Cbg14GN7j4K9AOXhCxMJC1Z7GRVD17SMNc0yWLPJZoPX/yYf0+4HpHUNWaxk5VGjeAluJIC3syuBZ4FPACMxzc7CnipdBMTNNlw+jtZfUl0HtiJcaipTXXdkh+ljuA3Ame46xxjUmVGopOOpb6TtfjTrEuWp7puyY9Sd7I+DKwJWYhIJuKAz2QnK6hNI0GVOoJfBfzSzO4jOpk2AO7+2iBViaRlOA741KdJ6qxOEl6pAX9FyCJEMjMSjaD7svigE2gEL0GVFPDu/kMzOwXY4O63mVkjoD1DUvkyG8HrxNsSXqnHovkT4AbgC/FN64FvhypKJDWFnawawUsVKnUn6zuAc4EeAHd/ElgdqiiR1Axns5O1Z7IHr/OySjilBvywu48Uvok/7KQpk1L54h586i0anfRDUlBqwP/QzD5EdPLt3wa+DtwYriyRlAxn06IZYjFYrVo0ElSpAb8J6AQeAv4UuAX4m1BFiaRmpI9xNwapT3nFBg2tmiYpQZU6i2bCzL4NfNvdOwPXJJKe4b64/27pr7u+RSN4CWrWEbxFrjCzg8BjwONm1mlmf5tOeSKBjfSmfqCxSfVL1YOXoOZq0byHaPbMC919pbuvAH4NONfM3hu8OpHQhvsY8LTbMzGN4CWwuQL+jcDr3X174QZ33wZcFt8nUtlG+o7OSU9bQ6umSUpQcwV8nbsfnHpj3Ievm2Z5kcoy3Ed/yif7mFTfqhaNBDVXwI8c530ilWGkL8MevFo0EtZcs2jONLPphhgGKU8cFglhuJc+lmaz7sI0SXewDGbxSNWbNeDdXQcUk+o2kmWLpgUmRmFsGOo0XpLklfpBJ5HqNJzhTtb61rgG9eElDAW85Nf4KIwPZzeCL5yqb7A7m/VL1VPAS37FOzgz28laCPiBQ9msX6qeAl7yK6NjwU9qXBldDh7OZv1S9RTwkl8Znc2p4CWf3QLA+6+9M5P1S/VTwEt+jWRzso+Cw7QAsALNhZcwFPCSX3EPvi+jnaxD1DPkdSw3BbyEoYCX/JocwWe0k5VoFL+cvszWL9VNAS/5ldHZnIp1eYtG8BKMAl7yayTbnawAXd7MCgW8BKKAl/yanAef4QieFpZrJ6sEooCX/Brpg5pFDGd45OvD3sJyUw9ewggW8GZ2kpndaWaPmtkjZvbuUOsSOS7DfbC4mUzOxxrrooWl9MP4WGY1SPUKOYIfA97n7s8FXgy8w8zOCLg+kfkZ6YuO6JihLm+hxhyGdDwaSV6wgHf3fe7+8/h6L/AosD7U+kTmbbi3LAIe0PFoJIhUevBm1g6cDdybxvpESjJSaNFkp/BpVgZ0PBpJXvCAN7Nm4BvAe9z9GQe+NrPLzWyzmW3u7OwMXY7IUcN9UJ9twGsELyEFDXgzqyMK9/9w929Ot4y7X+XuG919Y1tbW8hyRI5VBiP4Lo/Xr4CXAELOojHgX4FH3f1TodYjctyGs9/JerRFo4CX5IUcwZ8L/BFwnpk9EH9dGHB9IvMz0pv5CH6IegZ9sY4JL0HMetLthXD3/yLLCcYis3Evix48QBfNLOnXCF6Sp0+ySj6NDYGPZz6CBzjkrdCvCQaSPAW85FN8JMmse/AAB30p9B/IugypQgp4yaeR+ABfZTCCjwL+YNZlSBVSwEs+TY7gyyDgWRq1aNyzLkWqjAJe8ik+FnzZjODHR3Q8GkmcAl7yqax68K3RlT7taJVkKeAln8qpB8/S6Ip2tErCFPCST+XUg/c44PsU8JIsBbzkU7n14EFz4SVxCnjJp+HyCfguWsBqNIKXxCngJZe+cNsWBn0x7R/+ftalMEENNK5SD14Sp4CXXGpmiD4asi7jqObV+rCTJE4BL7nUZIP0+5KsyziqqU0tGkmcAl5yqYkh+stuBK+Al2Qp4CWXohZNuY3gdbgCSZYCXnIpatGU2Qh+bPDo9E2RBCjgJZfKrkXTtDq6VB9eEqSAl1xqtkH6ym0nK+jDTpIoBbzkUguD9NKYdRlHNccBrxG8JEgBL/kzPkqjDdNbViP4uEWjmTSSIAW85M9wdCTJshrBN7UBphG8JEoBL/kzdASAXi+jgK9dBM0nQM/erCuRKqKAl/wZ7gGgt5zmwQO0rlXAS6IU8JI/Q4WAL6MRPEDreujdl3UVUkUU8JI/cYump5xaNAAta6FnT9ZVSBVRwEv+DJfrCH5t9MdnZCDrSqRKKOAlfwotmnKaJglRiwbUppHEKOAlf8p1BN+yNrpUm0YSsijrAkRSN3SEQV/MWBn9+LdvupnTbC931AM9GsFLMjSCl/wZ7im/0Tuw31dEV3o1VVKSoYCX/BnqKb/+OzBAQzSzR3PhJSEKeMmfMh3BA+zzFQp4SYwCXvJnqKf85sDHdnsbdO/MugypEgp4yZ/hHnrKdAT/tK+Grp06dZ8kQgEv+TPUU14HGiuyy1dH0zgHu7IuRaqAAl7yp4x78Ls8PvFH1/ZsC5GqoICXfBkfhdGBspxFA3GLBqI2jcgCKeAlX8rxZB9Fdk0G/I5M65DqoICXfBnqBsrsZB9FBmiAxlUKeEmEAl7yZTAK+G6aMi5kFsvbNVVSEhEs4M3s38zsgJk9HGodIvMWz07p9uaMC5nF8nY4rJ2ssnAhR/DXABcEfH6R+SsEPGUc8CtOgyO7YGw460qkwgULeHf/EXA41POLHJc44I+U8wh+1QbwCY3iZcHUg5d8iXvwR8q5B7/yV6LLQ09mW4dUvMwD3swuN7PNZra5s7Mz63Kk2g12weJmRsvoWPDPUAj4gwp4WZjMA97dr3L3je6+sa2tLetypNoNdsGS5VlXMbuGVmheA4eeyroSqXCZB7xIqga7YMmyrKuY28pfUcDLgoWcJvkV4KfAs81st5m9NdS6REpWCSN4gFW/ohaNLFiwRqS7vz7Uc4sct8EuaHt21lXMqn3Tzby1dpSP1B2GgcPQuCLrkqRCqUUj+TLUXREj+G2+LrqiUbwsgAJe8sO9Ylo023xtdEVTJWUBFPCSHyP9MD5SEQG/29sY8VqN4GVBFPCSH/3x5yyayn867ji17HRNlZSFUcBLfvQfjC4rIOAhbtNoBC8LoICX/Jgcwa/Kto4Sbfe1cHgbjI9lXYpUKAW85EcFtWgAtvpamBiFI09nXYpUKAW85MdAoUVTGSP4bRPxTJqD6sPL8VHAS370H4TFLVBXnifcnkpTJWWhFPCSH/2dFTN6B+iiNZrSqR2tcpwU8JIf/Z0V03+ftHKDpkrKcVPAS370H6y8gF+1QSN4OW4KeMmP/k5oWpl1FfOz6nTo2z95JiqR+VDASz6Mj0UB33xC1pXMz+rnRpedj2Vbh1QkBbzkQ19HdCLr1vVZVzI/bc+JLg/8Mts6pCIp4CUfevZGl63rsq1jvpaeBIub4YBG8DJ/CnjJh94o4F/9xa20b7o542JK1/6h7/HA0An85Kf/lXUpUoEU8JIP8Qh+n1fe2ZGemDiR02t2ZV2GVCAFvORDzx6GvI5umrOuZN4e9xNps56jR8MUKZECXvKhZ288eresK5m3x/zk6Mq+LdkWIhVHAS/50LOXDiqvPQPw4MSzmHCDPfdnXYpUGAW85MOR3RXZfwfopZGtvg52b866FKkwCnipfmPDcGQ3O73CPuRU5IGJZ8GezdGJw0VKpICX6tf9NODsmFiTdSXH7X4/HQYOQefjWZciFUQBL9Xv8DYAnvbVGRdy/O4aPzO68sT3si1EKooCXqpfHPA7vHJH8PtZCWvPhMcV8FI6BbxUv8PboL6Vw7RkXcnCPOdi2HWfTuEnJVPAS/U7vA1WnEolzoE/xjlvhNo6+NE/ZF2JVAgFvFS/A49FZ0aqcO0f38znhl8ND14Pj/9n1uVIBVDAS3UbOAw9u2HN87OuJBGfHXsdj0ycAjf8Mez5edblSJlTwEt163gkujzhBdnWkZAR6njzyPujM1P9++/A9h9nXZKUMQW8VLeOh6PLKhnBA3SyHN58C7SsgeteB//1aRgfzbosKUMKeKlu+x+GxlWVd6q+ObRf+SBn7n4f/zlyJtx2BfzLeZpdI8+ggJfqtuteWH8OWIXPoJnGEZp5++h7+dOR98KR3fDFC+DQ1qzLkjKigJfq1bsfDj0J7S/NupKgvj/xQn6r+4Mc6hvi0X+8JDr2jggKeKlmO6LT3L3mJquo0/Qdj62+nr8afTvPrdkFP/xk1uVImVDAS/Xadic93sgvvT3rSlJx58TZ3DD+m/CTzxydPSS5poCX6jQ+Co/exG0T5zCRox/zj42+ARqWwnffBRPjWZcjGcvPT77ky+Pfg6Fubhz/9awrSVU3Lby7+w9gz2b+/iPvzrocyZgCXqqPO9z9f2HZyfxw4sysq0nddybO5bbxs/nQoi/DEz/IuhzJkAJeqs8j34Td98FL35er9sxRxntH38ETfiJ85VK49e9g34PQtQOO7IH+Q5ppkxPmAU8BZmYXAP8I1AJXu/uVsy2/ceNG37xZ552UBeh+Gq56ObSuh8vvov1D+T0oVxODfLTui/xuzU+osam/5wbL22HD+fDCt0Hb6VmUKAkws/vdfeO094UKeDOrBZ4AfhvYDfwMeL27/3KmxyjgZUG6dsJ1r6Pn4B4uGfkY231t1hWVhfV0clbNVpbYMIsYZzGjrLBezrCdvKxmC/U1E/CC34eXfQBWPivrcmWeZgv4RQHX+yLgKXffFhdxPXAJMGPAV43iP5qT16e7rYTb57NsqesrfG8GtYujr5raaV5ISnyW1zr19bjDaD8MHYHBbhjqjq53PAI//X9gNbxl5P0K9yJ7aGPPRNu0962gh8sX3cSbtnyLui1fZ9H6M2HVs2HpibDspOiydT20rI1m5xR/InhsBIZ7ob8T+g9El32d0W1Nq6Jj5bSsiR7b1Jbtz1hOhQz49cCuou93A78WZE2ffBaMDpJKiM60bKWzmijorfBLOEPATnfb8S6TsFvHz+Hvx97I7go+92raDtPKlWN/yL+OXcgfLfoBv7rrSU7ZcxtrOMwim3jmA2oWRT8jPg4TY6WvyGpgUcNcC81yV/UdauIYTavg3VsSf9qQAT/dO/KM324zuxy4PP62z8yO97Txq4CDx/nYkFTX/CygrrviryCqcHsdtRNIuDla1dsreXtX8R473rpOmemOkAJcQnAAAAZlSURBVAG/Gzip6PsTgb1TF3L3q4CrFroyM9s8Ux8qS6prflTX/Kiu+clbXSHnkP0M2GBmp5rZYuBS4LsB1yciIkWCjeDdfczM3gl8n2ia5L+5uw6QISKSkpAtGtz9FuCWkOsosuA2TyCqa35U1/yorvnJVV1BP+gkIiLZyePnuEVEcqGiAt7MVpjZrWb2ZHy5fIblxs3sgfjru0W3n2pm98aP/2q88zeVuszsLDP7qZk9YmYPmtkfFN13jZltL6r5rAXWc4GZPW5mT5nZpmnur49f/1Px9mgvuu+D8e2Pm9mrFlLHPGv6SzP7ZbxtbjezU4rum/b9TLG2N5tZZ1ENbyu6703x+/6kmb0p5bo+XVTTE2bWXXRfkG1mZv9mZgfM7OEZ7jcz+2xc84Nmdk7RfSG31Vx1vSGu50Ezu9vMziy6b4eZPRRvq0Rni5ZQ18vN7EjRe/W3RffN+v6XxN0r5gv4JLApvr4J+N8zLNc3w+1fAy6Nr38e+LO06gJOBzbE19cB+4Bl8ffXAP89oVpqga3AacBiYAtwxpRl/hz4fHz9UuCr8fUz4uXrgVPj56lNqaZXAI3x9T8r1DTb+5ni9noz8LlpHrsC2BZfLo+vL0+rrinL/wXRRIag2wz4TeAc4OEZ7r8Q+B7R52BeDNwbeluVWNdLCusDXl2oK/5+B7Aqo+31cuCmhb7/M31V1Aie6FAHX4qvfwn4nVIfaGYGnAfccDyPX2hd7v6Euz8ZX98LHACm//z4wkweIsLdR4DCISJmqvcG4Lfi7XMJcL27D7v7duCp+PmC1+Tud7r7QPztPUSfm0hDKdtrJq8CbnX3w+7eBdwKXJBRXa8HvpLQumfk7j8CDs+yyCXAv3vkHmCZma0l7Laasy53vzteL6T481XC9prJQn4uJ1VawJ/g7vsA4suZPpPeYGabzeweMyuE7Uqg290Ln6/eTXQ4hTTrAsDMXkT0V3lr0c0fj/99/LSZ1S+glukOETH1dU4uE2+PI0Tbp5THhqqp2FuJRoEF072fSSm1tt+L358bzKzwAb5Q22tezx23s04F7ii6OeQ2m81MdYfcVvM19efLgR+Y2f0WfbI+bb9uZlvM7Htm9rz4tkS2V9BpksfDzG4D1kxz14fn8TQnu/teMzsNuMPMHgJ6plmu5ClECdVFPJq5FniTuxcO9vFBYD9R6F8FfAD46Hyet3gV09w2zbFip12mpMNLHIeSn9fMLgM2Ai8ruvkZ76e7b53u8YFquxH4irsPm9nbif77Oa/Ex4asq+BS4AZ3Lz5HX8htNpu0f7bmxcxeQRTwv1F087nxtloN3Gpmj8Uj7zT8HDjF3fvM7ELg28AGEtpeZTeCd/dXuvvzp/n6DtARB2QhKA/M8Bx748ttRAcnOZvo+BPLzKzwR23aQyeErMvMWoGbgb+J/30tPPe++F/aYeCLLKwtUsohIiaXibfHUqJ/I0s6vESgmjCzVxL9wXxtvC2AGd/PpMxZm7sfKqrnX4BfLfWxIesqcilT2jOBt9lsZqo75LYqiZn9N+Bq4BJ3P1S4vWhbHQC+RTJtyZK4e4+798XXbwHqzGwVSW2vpHcqhPwC/oFjd2Z+cppllgP18fVVwJPEOyeAr3PsTtY/T7GuxcDtwHumuW9tfGnAZ4ArF1DLIqIdWKdydOfM86Ys8w6O3cn6tfj68zh2J+s2ktnJWkpNZxO1rDaU+n4m9N6VUtvaouu/C9wTX18BbI9rXB5fX5FWXfFyzybaSWgpbrN2Zt5peBHH7mS9L/S2KrGuk4n2Kb1kyu1NQEvR9buBC1Ksa03hvSP6w/J0vO1Kev/nXHeSLyT0F1Gf+Pb4B/b2wg8I0b/0V8fXXwI8FG+Qh4C3Fj3+NOC++I3+euGXIKW6LgNGgQeKvs6K77sjrvVh4DqgeYH1XEh0spWtwIfj2z5KNDIGaIhf/1Px9jit6LEfjh/3OPDqBN+7uWq6Dego2jbfnev9TLG2TwCPxDXcCTyn6LF/HG/Hp4C3pFlX/P0VTBkQhNxmRP8p7It/lncTtTveDrw9vt+Af4prfgjYmNK2mquuq4Guop+vzfHtp8XbaUv8Hn845breWfSzdQ9Ff4Cme//n+6VPsoqIVKmy68GLiEgyFPAiIlVKAS8iUqUU8CIiVUoBLyJSpRTwIiJVSgEvIlKlFPAiIlXq/wO7fJ+CEhfysgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "label_cols = ['fin_temperature'] # This can be changed to other column name\n",
    "ok_std = 2 # cut-off SD value\n",
    "\n",
    "fig=plt.figure(figsize=(6,5))\n",
    "ax = fig.add_subplot(111)\n",
    "train_scaled[label_cols].plot.hist(bins=50, ax=ax, normed=True)\n",
    "train_scaled[label_cols].plot.density(ax=ax)\n",
    "plt.title(label_cols)\n",
    "ax.get_legend().remove()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.07861215782918889"
      ]
     },
     "execution_count": 106,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "label_mean = train_scaled[label_cols].mean().values[0]\n",
    "label_std = train_scaled[label_cols].std().values[0]\n",
    "label_std"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>asm_Force</th>\n",
       "      <th>asm_Length</th>\n",
       "      <th>asm_Height</th>\n",
       "      <th>asm_Press-fit</th>\n",
       "      <th>asm_force</th>\n",
       "      <th>asm_Length</th>\n",
       "      <th>asm_Depth</th>\n",
       "      <th>asm_depth</th>\n",
       "      <th>asm_Depth</th>\n",
       "      <th>asm_parallelism</th>\n",
       "      <th>...</th>\n",
       "      <th>int_Shape_Ascending</th>\n",
       "      <th>int_Shape_V-shape</th>\n",
       "      <th>int_Shape_Descending</th>\n",
       "      <th>int_Shape_Flat</th>\n",
       "      <th>fin_Shape_V-shape</th>\n",
       "      <th>fin_Shape_Ascending</th>\n",
       "      <th>fin_Shape_Descending</th>\n",
       "      <th>fin_Shape_Flat</th>\n",
       "      <th>label</th>\n",
       "      <th>mean_diff</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>0.347826</td>\n",
       "      <td>0.997179</td>\n",
       "      <td>0.523248</td>\n",
       "      <td>0.807539</td>\n",
       "      <td>0.555048</td>\n",
       "      <td>0.994274</td>\n",
       "      <td>0.595506</td>\n",
       "      <td>0.560773</td>\n",
       "      <td>0.991176</td>\n",
       "      <td>0.110599</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>0.469565</td>\n",
       "      <td>0.998055</td>\n",
       "      <td>0.519205</td>\n",
       "      <td>0.806525</td>\n",
       "      <td>0.566210</td>\n",
       "      <td>0.994641</td>\n",
       "      <td>0.713483</td>\n",
       "      <td>0.668508</td>\n",
       "      <td>0.994759</td>\n",
       "      <td>0.211982</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>0.443478</td>\n",
       "      <td>0.997568</td>\n",
       "      <td>0.519373</td>\n",
       "      <td>0.806715</td>\n",
       "      <td>0.524099</td>\n",
       "      <td>0.994641</td>\n",
       "      <td>0.573034</td>\n",
       "      <td>0.477901</td>\n",
       "      <td>0.990695</td>\n",
       "      <td>0.211982</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>0.426087</td>\n",
       "      <td>0.996693</td>\n",
       "      <td>0.519879</td>\n",
       "      <td>0.806208</td>\n",
       "      <td>0.547438</td>\n",
       "      <td>0.994641</td>\n",
       "      <td>0.699438</td>\n",
       "      <td>0.640884</td>\n",
       "      <td>0.993743</td>\n",
       "      <td>0.170507</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>0.391304</td>\n",
       "      <td>0.998055</td>\n",
       "      <td>0.521900</td>\n",
       "      <td>0.807856</td>\n",
       "      <td>0.538305</td>\n",
       "      <td>0.994641</td>\n",
       "      <td>0.744382</td>\n",
       "      <td>0.632597</td>\n",
       "      <td>0.995508</td>\n",
       "      <td>0.336406</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 84 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   asm_Force  asm_Length  asm_Height  asm_Press-fit  asm_force  asm_Length  \\\n",
       "0   0.347826    0.997179    0.523248       0.807539   0.555048    0.994274   \n",
       "1   0.469565    0.998055    0.519205       0.806525   0.566210    0.994641   \n",
       "2   0.443478    0.997568    0.519373       0.806715   0.524099    0.994641   \n",
       "3   0.426087    0.996693    0.519879       0.806208   0.547438    0.994641   \n",
       "4   0.391304    0.998055    0.521900       0.807856   0.538305    0.994641   \n",
       "\n",
       "   asm_Depth  asm_depth  asm_Depth  asm_parallelism  ...  int_Shape_Ascending  \\\n",
       "0   0.595506   0.560773   0.991176         0.110599  ...                  1.0   \n",
       "1   0.713483   0.668508   0.994759         0.211982  ...                  0.0   \n",
       "2   0.573034   0.477901   0.990695         0.211982  ...                  0.0   \n",
       "3   0.699438   0.640884   0.993743         0.170507  ...                  0.0   \n",
       "4   0.744382   0.632597   0.995508         0.336406  ...                  0.0   \n",
       "\n",
       "   int_Shape_V-shape  int_Shape_Descending  int_Shape_Flat  fin_Shape_V-shape  \\\n",
       "0                0.0                   0.0             0.0                1.0   \n",
       "1                1.0                   0.0             0.0                1.0   \n",
       "2                1.0                   0.0             0.0                1.0   \n",
       "3                1.0                   0.0             0.0                0.0   \n",
       "4                0.0                   1.0             0.0                0.0   \n",
       "\n",
       "   fin_Shape_Ascending  fin_Shape_Descending  fin_Shape_Flat  label  mean_diff  \n",
       "0                  0.0                   0.0             0.0      1          1  \n",
       "1                  0.0                   0.0             0.0      1          1  \n",
       "2                  0.0                   0.0             0.0      1          1  \n",
       "3                  1.0                   0.0             0.0      1          1  \n",
       "4                  1.0                   0.0             0.0      1          1  \n",
       "\n",
       "[5 rows x 84 columns]"
      ]
     },
     "execution_count": 117,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_scaled['label'] = (np.abs(train_scaled[label_cols] - label_mean) < ok_std*label_std)*1\n",
    "test_scaled['label'] = (np.abs(test_scaled[label_cols] - label_mean) < ok_std*label_std)*1\n",
    "train_scaled.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'Number of observations per bin')"
      ]
     },
     "execution_count": 119,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEICAYAAABS0fM3AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAZgklEQVR4nO3de5hV9X3v8fcHRBBFNANaBAmUEhOrxstUY/QkmqgB7fFWatXQJxpP8DRabYwe4dQQL7El9WiMOUZDKtXEGGuIRhoxUlM0Gi/cRAOIBZGUEY8iUQQVRPyeP/Ya3Q4ze34Ms9aGvT6v59nPrNte+7N9Rr6z1u+yFBGYmVl59ah3ADMzqy8XAjOzknMhMDMrORcCM7OScyEwMys5FwIzs5LbIa8TS5oC/DnwSkTs185+Ad8FjgfeAs6KiHmdnXfAgAExbNiwbk5rZtbY5s6d+2pEDGxvX26FALgV+L/AjzrYPxoYmb0OA27KftY0bNgw5syZ000RzczKQdLvO9qX262hiPgN8Icah5wE/CgqngB2kzQorzxmZta+erYRDAZWVK23ZNvMzKxA9SwEamdbu/NdSBonaY6kOatWrco5lplZueTZRtCZFmDvqvUhwMr2DoyIycBkgObm5s2KxcaNG2lpaWH9+vV55Nxm9OnThyFDhtCrV696RzGzBlLPQjANOF/SnVQaiddExEtdOVFLSwv9+vVj2LBhVDojNZ6IYPXq1bS0tDB8+PB6xzGzBpJn99GfAkcBAyS1AN8EegFExM3AdCpdR5dS6T56dlc/a/369Q1dBAAk0dTUhG+NmVl3y60QRMQZnewP4Lzu+rxGLgKtyvAdzax4HlncDV5//XW+//3vb/H7jj/+eF5//fUcEpmZpatnG0Fuho2/r1vPt3zSCTX3txaCr371qx/avmnTJnr27Nnh+6ZPn94t+cwa0uX9650gzeVr6p1gqzVkISja+PHjef755znwwAPp1asXu+yyC4MGDWL+/PksWrSIk08+mRUrVrB+/XouvPBCxo0bB3wwSnrdunWMHj2aI488kscee4zBgwdz7733stNOO9X5m1kj6u4/lPKyvE+9E5SHbw11g0mTJjFixAjmz5/PNddcw6xZs7j66qtZtGgRAFOmTGHu3LnMmTOHG264gdWrV292jiVLlnDeeeexcOFCdtttN37+858X/TXMrKR8RZCDQw899ENdPG+44QbuueceAFasWMGSJUtoamr60HuGDx/OgQceCMAhhxzC8uXLC8trZuXmQpCDnXfe+f3lhx56iAcffJDHH3+cvn37ctRRR7U78K13797vL/fs2ZO33367kKxmZr411A369evH2rVr2923Zs0adt99d/r27cvixYt54oknCk5nZlabrwi6QVNTE0cccQT77bcfO+20E3vuuef7+0aNGsXNN9/MAQccwD777MOnPvWpOiY1M9tcQxaCzrp75uGOO+5od3vv3r25//77293X2g4wYMAAFixY8P72iy++uNvzmZl1xLeGzMxKzoXAzKzkXAjMzErOhcDMrORcCMzMSs6FwMys5FwIukFXp6EGuP7663nrrbe6OZGZWbqGHEfQ7dPXdjLNbEfTUKe4/vrrGTt2LH379u1qOjOzrdKYhaBg1dNQH3vsseyxxx7cddddbNiwgVNOOYUrrriCN998k9NOO42WlhY2bdrEN77xDV5++WVWrlzJ0UcfzYABA5g5c2a9v4qZlZALQTeYNGkSCxYsYP78+cyYMYOpU6cya9YsIoITTzyR3/zmN6xatYq99tqL++6rzAW/Zs0a+vfvz3XXXcfMmTMZMGBAnb+FmZWV2wi62YwZM5gxYwYHHXQQBx98MIsXL2bJkiXsv//+PPjgg1x66aU88sgj9O+/nTx9ycwanq8IullEMGHCBM4999zN9s2dO5fp06czYcIEjjvuOCZOnFiHhGZmH+Yrgm5QPQ31F77wBaZMmcK6desAePHFF3nllVdYuXIlffv2ZezYsVx88cXMmzdvs/eamdVD0hWBpE8Dw6qPj4gf5ZRpu1M9DfXo0aM588wzOfzwwwHYZZdduP3221m6dCmXXHIJPXr0oFevXtx0000AjBs3jtGjRzNo0CA3FptZXSgiah8g/RgYAcwHNmWbIyIuyDlbu5qbm2POnDkf2vbss8/yiU98oh5xClem72r52H4eXn9mvSOk6aR7+bZC0tyIaG5vX8oVQTOwb3RWMczMbLuU0kawAPijvIOYmVl9pFwRDAAWSZoFbGjdGBEn5pbKzMwKk1IILs87RHeICCTVO0aufHfOzPLQaSGIiIeLCLI1+vTpw+rVq2lqamrYYhARrF69mj59+tQ7ipk1mA4LgaRHI+JISWuB6j9FRaXX0K65p0s0ZMgQWlpaWLVqVb2j5KpPnz4MGTKk3jHMrMF0WAgi4sjsZ7/i4nRNr169GD58eL1jmJltl1IHlB0MHEnlyuDRiHgq11RmZlaYTruPSpoI3AY0UelBdKuky/IOZmZmxUi5IjgDOCgi1gNImgTMA76VZzAzMytGyoCy5UB1V5XewPMpJ5c0StJzkpZKGt/O/qGSZkp6StIzko5PSm1mZt2mVq+h71FpE9gALJT079n6scCjnZ1YUk/gxuz4FmC2pGkRsajqsMuAuyLiJkn7AtOpTG5nZmYFqXVrqHVmt7nAPVXbH0o896HA0ohYBiDpTuAkoLoQBNDaDbU/sDLx3GZm1k1qdR+9bSvPPRhYUbXeAhzW5pjLgRmS/hbYGTimvRNJGgeMAxg6dOhWxjIzs2p5PpimvSG+bedIOAO4NSKGAMcDP5a0WaaImBwRzRHRPHDgwByimpmVV56FoAXYu2p9CJvf+jkHuAsgIh6n0ijtp7ibmRWoZiGQ1FPSNV0892xgpKThknYETgemtTnmv4DPZ5/1CSqFoLHniTAz28bULAQRsQk4RF2YyS0i3gXOBx4AnqXSO2ihpCsltU5h/XXgK5KeBn4KnOUH4JiZFStlQNlTwL2Sfga82boxIu7u7I0RMZ1Kl9DqbROrlhcBRySnNTOzbpdSCD4CrAY+V7UtgE4LgZmZbftSnkdwdhFBzMysPlImnfuYpF9LWpCtH+BJ58zMGkdK99EfAhOAjQAR8QyVHkBmZtYAUgpB34iY1Wbbu3mEMTOz4qUUglcljSAbFSxpDPBSrqnMzKwwKb2GzgMmAx+X9CLwAvDFXFOZmVlhUnoNLQOOkbQz0CMi1uYfy8zMipLSa6hJ0g3AI8BDkr4rqSn/aGZmVoSUNoI7qcz/8xfAmGz5X/MMZWZmxUkaWRwRV1Wtf0vSyXkFMjOzYqVcEcyUdLqkHtnrNOC+vIOZmVkxUgrBucAdVJ5dvIHKraKLJK2V9Eae4czMLH8pvYb6FRHEzMzqI88nlJmZ2XbAhcDMrORcCMzMSi5lQNkISb2z5aMkXSBpt/yjmZlZEVKuCH4ObJL0J8AtwHAqvYjMzKwBpBSC97IH0Z8CXB8RXwMG5RvLzMyKklIINko6A/gS8MtsW6/8IpmZWZFSCsHZwOHA1RHxgqThwO35xjIzs6KkDChbBFxQtf4CMCnPUGZmVpxOC4GkI4DLgY9mxwuIiPjjfKOZmVkRUmYfvQX4GjAX2JRvHDMzK1pKIVgTEffnnsTMzOoipRDMlHQNcDeV2UcBiIh5uaUyM7PCpBSCw7KfzVXbAvhc98cxM7OipfQaOrqIIGZmVh8pcw31l3SdpDnZ61pJ/YsIZ2Zm+UsZUDYFWAuclr3eAP4lz1BmZlaclDaCERHxF1XrV0ian1cgMzMrVsoVwduSjmxdyQaYvZ1fJDMzK1LKFcHfALdl7QIC/gCclWcoMzMrTqdXBBExPyI+CRwA7B8RB0XE0yknlzRK0nOSlkoa38Exp0laJGmhJD/nwMysYB1eEUgaGxG3S7qozXYAIuK6WieW1BO4ETgWaAFmS5qWTWLXesxIYAJwRES8JmmPLn8TMzPrklq3hnbOfvZrZ18knPtQYGlELAOQdCdwErCo6pivADdGxGsAEfFKwnnNzKwbdVgIIuIH2eKDEfHb6n1Zg3FnBgMrqtZb+GCUcquPZef7LdATuDwiftX2RJLGAeMAhg4dmvDRZmaWKqXX0PcSt7Wldra1vZLYARgJHAWcAfyzpN02e1PE5IhojojmgQMHJny0mZmlqtVGcDjwaWBgm3aCXan89d6ZFmDvqvUhwMp2jnkiIjYCL0h6jkphmJ1wfjMz6wa1rgh2BHahUiz6Vb3eAMYknHs2MFLScEk7AqcD09oc8wvgaABJA6jcKlq2JV/AzMy2Tq02goeBhyXdGhG/39ITR8S7ks4HHqByBTElIhZKuhKYExHTsn3HSVpE5aE3l0TE6i59EzMz65KUAWVvZc8j+FOgT+vGiOh0GuqImA5Mb7NtYtVyABdlLzMzq4OUxuKfAIuB4cAVwHJ8D9/MrGGkFIKmiLgF2BgRD0fEl4FP5ZzLzMwKknJraGP28yVJJ1Dp+TMkv0hmZlaklELwrWzCua9TGT+wK/C1XFOZmVlhUgrBkxGxBlhD1tXTzMwaR0obwWOSZkg6R9LuuScyM7NCpUxDPRK4jEr30bmSfilpbO7JzMysEClXBETErIi4iMqMon8Abss1lZmZFabTQiBpV0lfknQ/8BjwEpWCYGZmDSClsfhpKnMCXRkRj+ecx8zMClazEGRPGbsnuy1kZmYNqOatoYjYBHyyoCxmZlYHKbeG5kuaBvwMeLN1Y0TcnVsqMzMrTEoh+AiwGqiebTQAFwIzswbQaSGIiLOLCGJmZvWR0n30Y5J+LWlBtn6ApMvyj2ZmZkVIGVD2Q2AC2SykEfEMlcdOmplZA0gpBH0jYlabbe/mEcbMzIqXUghelTSCSgMxksZQGV1sZmYNIKXX0HnAZODjkl4EXgA86ZyZWYNI6TW0DDhG0s5Aj4hYm38sMzMrSkqvoQsl7Qq8BXxH0jxJx+UfzczMipDSRvDliHgDOA7YAzgbmJRrKjMzK0xKIVD283jgXyLi6aptZma2nUspBHMlzaBSCB6Q1A94L99YZmZWlJReQ+cABwLLIuItSU1Ubg+ZmVkDSOk19J6kYcBYSQE8GhH35B3MzMyKkdJr6PvA/wR+BywAzpV0Y97BzMysGCm3hj4L7BcRrSOLb6NSFMzMrAGkNBY/BwytWt8beCafOGZmVrQOrwgk/RuV+YX6A89Kap147lDgsQKymZlZAWrdGvo/haUwM7O66bAQRMTDrcuS9gT+LFudFRGv5B3MzMyKkdJr6DRgFvCXwGnAk9lU1J2SNErSc5KWShpf47gxkkJSc2pwMzPrHim9hv4e+LPWqwBJA4EHgam13iSpJ3AjcCzQAsyWNC0iFrU5rh9wAfDklsc3M7OtldJrqEebW0GrE993KLA0IpZFxDvAncBJ7Rx3FfBPwPqEc5qZWTdL+Qf9V5IekHSWpLOA+4DpCe8bDKyoWm/Jtr1P0kHA3hHxy8S8ZmbWzVKmmLhE0qnAkVRmHZ2cOMVEezOUxvs7pR7Ad4CzOj2RNA4YBzB06NBOjjYzsy2R0kZARNwN3L2F526hMvis1RBgZdV6P2A/4CFJAH8ETJN0YkTMafP5k6k8LpPm5ubAzMy6Tcqtoa6aDYyUNFzSjsDpwLTWnRGxJiIGRMSwiBgGPAFsVgTMzCxfuRWCiHgXOB94AHgWuCsiFkq6UtKJeX2umZltmVpTTPw6Ij4v6dsRcWlXTh4R02nTsBwREzs49qiufIaZmW2dWm0EgyR9FjhR0p20afyNiHm5JjMzs0LUKgQTgfFUGnmva7MvgM/lFcrMzIpTa66hqcBUSd+IiKsKzGRmZgVKGUdwVda4+5ls00MeAGZm1jhSJp37R+BCYFH2ujDbZmZmDSBlQNkJwIER8R68/6jKp4AJeQYzM7NipI4j2K1quX8eQczMrD5Srgj+EXhK0kwqXUg/g68GzMwaRkpj8U8lPUTlCWUCLo2I/5d3MDMzK0bqpHMvUTVPkJmZNY48J50zM7PtgAuBmVnJ1SwEknpIWlBUGDMzK17NQpCNHXhakh8LZmbWoFIaiwcBCyXNAt5s3RgRfqaAmVkDSCkEV+SewszM6iZlHMHDkj4KjIyIByX1BXrmH83MzIqQMuncV4CpwA+yTYOBX+QZyszMipPSffQ84AjgDYCIWALskWcoMzMrTkoh2BAR77SuSNqByhPKzMysAaQUgocl/W9gJ0nHAj8D/i3fWGZmVpSUQjAeWAX8DjgXmA5clmcoMzMrTkqvofeyh9E8SeWW0HMR4VtDZmYNotNCIOkE4GbgeSrTUA+XdG5E3J93ODMzy1/KgLJrgaMjYimApBHAfYALgZlZA0hpI3iltQhklgGv5JTHzMwK1uEVgaRTs8WFkqYDd1FpI/hLYHYB2czMrAC1bg3996rll4HPZsurgN1zS2RmZoXqsBBExNlFBjEzs/pI6TU0HPhbYFj18Z6G2sysMaT0GvoFcAuV0cTv5RvHzMyKllII1kfEDbknMTOzukgpBN+V9E1gBrChdWNEzMstlZmZFSalEOwP/DXwOT64NRTZupmZbedSCsEpwB9XT0WdStIo4LtUnmj2zxExqc3+i4D/AbxLpVvqlyPi91v6OWZm1nUpI4ufBnbb0hNL6gncCIwG9gXOkLRvm8OeApoj4gAqT0H7py39HDMz2zopVwR7AoslzebDbQSddR89FFgaEcsAJN0JnAQsqjrHzKrjnwDGJuY2M7NuklIIvtnFcw8GVlSttwCH1Tj+HDyRnZlZ4VKeR/BwF8+t9k7X7oHSWKCZD6axaLt/HDAOYOjQoV2MY2Zm7em0jUDSWklvZK/1kjZJeiPh3C3A3lXrQ4CV7Zz/GODvgRMjYkPb/QARMTkimiOieeDAgQkfbWZmqVKuCPpVr0s6mcr9/87MBkZmU1S8CJwOnNnmXAcBPwBGRYSntjYzq4OUXkMfEhG/IGEMQUS8C5wPPAA8C9wVEQslXSmptaH5GmAX4GeS5kuatqV5zMxs66RMOndq1WoPKvfyk55ZHBHTqTzsvnrbxKrlY9JimplZXlJ6DVU/l+BdYDmVbqBmZtYAUtoI/FwCM7MGVutRlRM72gdERFyVQx4zMytYrSuCN9vZtjOVgV9NgAuBmVkDqPWoymtblyX1Ay4EzgbuBK7t6H1mZrZ9qdlGIOkjwEXAF4HbgIMj4rUigpmZWTFqtRFcA5wKTAb2j4h1haUyM7PC1BpQ9nVgL+AyYGXVNBNrE6eYMDOz7UCtNoItHnVsZmbbH/9jb2ZWci4EZmYl50JgZlZyLgRmZiXnQmBmVnIuBGZmJedCYGZWci4EZmYl50JgZlZyLgRmZiXnQmBmVnIuBGZmJedCYGZWci4EZmYl50JgZlZyLgRmZiXnQmBmVnIuBGZmJedCYGZWci4EZmYl50JgZlZyO9Q7QKMaNv6+ekdIsnzSCfWOYGZ15kJQdpf3r3eCNJevqXcCs4blW0NmZiXnQmBmVnK5FgJJoyQ9J2mppPHt7O8t6V+z/U9KGpZnHjMz21xuhUBST+BGYDSwL3CGpH3bHHYO8FpE/AnwHeDbeeUxM7P25XlFcCiwNCKWRcQ7wJ3ASW2OOQm4LVueCnxeknLMZGZmbeTZa2gwsKJqvQU4rKNjIuJdSWuAJuDV6oMkjQPGZavrJD2XS+ISEgygzX/vbdIV/vugbPy72e0+2tGOPAtBe/91ogvHEBGTgcndEco+TNKciGiudw6ztvy7WZw8bw21AHtXrQ8BVnZ0jKQdgP7AH3LMZGZmbeRZCGYDIyUNl7QjcDowrc0x04AvZctjgP+IiM2uCMzMLD+53RrK7vmfDzwA9ASmRMRCSVcCcyJiGnAL8GNJS6lcCZyeVx7rkG+52bbKv5sFkf8ANzMrN48sNjMrORcCM7OScyEwMys5F4KSkbSPpMMl9cqmATHbpvj3snhuLC4RSacC/wC8mL3mALdGxBt1DWYGSPpYRPxnttwzIjbVO1NZ+IqgJCT1Av4KOCciPg/cS2Uw3/+StGtdw1npSfpzYL6kOwAiYpOvDIrjQlAuuwIjs+V7gF8COwJnerI/qxdJOwPnA38HvCPpdnAxKJILQUlExEbgOuBUSf8tIt4DHgXmA0fWNZyVWkS8CXwZuAO4GOhTXQzqma0sXAjK5RFgBvDXkj4TEZsi4g5gL+CT9Y1mZRYRKyNiXUS8CpwL7NRaDCQdLOnj9U3Y2Pzw+hKJiPWSfkJlhtcJ2f9cG4A9gZfqGs4sExGrJZ0LXCNpMZUpao6uc6yG5kJQMhHxmqQfAouo/OW1HhgbES/XN5nZByLiVUnPUHnC4bER0VLvTI3M3UdLLGuIi6y9wGybIWl34C7g6xHxTL3zNDoXAjPbJknqExHr652jDFwIzMxKzr2GzMxKzoXAzKzkXAjMzErOhcCsBknrOtk/TNKCLTznrZLGbF0ys+7jQmBmVnIuBGYJJO0i6deS5kn6naSTqnbvIOk2Sc9Imiqpb/aeQyQ9LGmupAckDapTfLOaXAjM0qwHTomIg6lMd3Bt1Yyt+wCTI+IA4A3gq9m0398DxkTEIcAU4Oo65DbrlKeYMEsj4B8kfQZ4DxhMZY4mgBUR8dts+XbgAuBXwH7Av2f1oieez8m2US4EZmm+CAwEDomIjZKWA32yfW1HZQaVwrEwIg4vLqJZ1/jWkFma/sArWRE4Gvho1b6hklr/wT+DynMengMGtm7PnhH9p4UmNkvkQmCW5idAs6Q5VK4OFlftexb4UjZb5keAmyLiHWAM8G1JT1N5ANCnC85slsRzDZmZlZyvCMzMSs6FwMys5FwIzMxKzoXAzKzkXAjMzErOhcDMrORcCMzMSs6FwMys5P4/pO8V0exFh04AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "t1 = train_scaled.groupby(['label'])['label'].count() / len(train_scaled)\n",
    "t2 = test_scaled.groupby(['label'])['label'].count() / len(test_scaled)\n",
    "\n",
    "tmp = pd.concat([t1, t2], axis=1)\n",
    "tmp.columns = ['train', 'test']\n",
    "tmp.plot.bar()\n",
    "plt.xticks(rotation=45)\n",
    "plt.ylabel('Number of observations per bin')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 130,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((4870, 81), (11364, 81))"
      ]
     },
     "execution_count": 130,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train = train_scaled.drop(columns=label_cols+['label'])\n",
    "X_test = test_scaled.drop(columns=label_cols+['label'])\n",
    "y_train = train_scaled['label']\n",
    "y_test = test_scaled['label']\n",
    "X_test.shape, X_train.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is obvious that this is imbalance dataset problem. It should be rebalanced before training by either oversampling, undersampling, or SMOTE. But due to time constraint, this has not yet happened."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 131,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=5, error_score='raise-deprecating',\n",
       "       estimator=SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0,\n",
       "  decision_function_shape='ovr', degree=3, gamma='scale', kernel='rbf',\n",
       "  max_iter=-1, probability=False, random_state=None, shrinking=True,\n",
       "  tol=0.001, verbose=False),\n",
       "       fit_params=None, iid='warn', n_jobs=None,\n",
       "       param_grid={'kernel': ('linear', 'rbf'), 'C': [1, 10]},\n",
       "       pre_dispatch='2*n_jobs', refit=True, return_train_score='warn',\n",
       "       scoring='accuracy', verbose=0)"
      ]
     },
     "execution_count": 131,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn import svm\n",
    "\n",
    "clf = svm.SVC(gamma='scale')\n",
    "param_grid = {\n",
    "    'kernel': ('linear', 'rbf'),\n",
    "    'C': [1:10]\n",
    "}\n",
    "\n",
    "grid_search = GridSearchCV(clf, param_grid, cv=5, scoring='accuracy')\n",
    "grid_search.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 157,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "from sklearn.metrics import accuracy_score\n",
    "from sklearn.metrics import confusion_matrix\n",
    "best = grid_search.best_estimator_\n",
    "y_pred = best.predict(X_test)\n",
    "cm = confusion_matrix(y_test, pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 177,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2, 2)"
      ]
     },
     "execution_count": 177,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cm.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 180,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Confusion matrix, without normalization\n",
      "[[   0  146]\n",
      " [   0 4724]]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAApcAAAFeCAYAAADQYzRwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3dd5glZZn38e9vQDISBWUGRRCRoCJJJImKCIqCqyiKEkRZVFQWwxpYFVfXsK4REyyIuC7BwAImxACKr5JBksAAIgMoEiUJznC/f1Q19sz09PQ050x1n/5+uOqaU0+lu/o059x9Vz31pKqQJEmSemFa1wFIkiRpcJhcSpIkqWdMLiVJktQzJpeSJEnqGZNLSZIk9YzJpSRJknpmya4DkCRJmoqWeOyTqmY/MK5t64G/nF5Vu/Q4pJ4wuZQkSepAzX6ApTd41bi2/dvFX1q9x+H0jMmlJElSJwIZvDsUTS4lSZK6ECDpOoqeM7mUJEnqygBWLgfvjCRJktQZK5eSJEld8bK4JEmSesMOPZIkSeolK5eSJEnqiWDlUpIkSb2SgaxcDl66LE1wSZZNclqSu5N8+1HsZ+8kP+llbF1Jsn2SqybK8ZKsk6SS+Af4PJL8IclO7ev3J/nvPhzjq0n+rdf7lbR4mFxKC5DktUnOT3JvkluS/CjJdj3Y9SuBNYHVqmrP8e6kqr5VVTv3IJ6+apO0p4y2TlX9qqo2WFwxzXu84QlTvyU5NslHF8ex+q2q/qOq3vho9pFkvyRnz7Pfg6rq3x9ddNIkkWnjmyawiR2d1JEkhwKfA/6DJhF8IvBlYPce7P5JwNVVNbsH+5r0rA72jz9baRJIxjdNYCaX0jySrAR8BHhrVX2vqu6rqr9X1WlV9e52naWTfC7Jze30uSRLt8t2TDIryTuT3NpWPfdvlx0OfBB4dVsRPSDJh5P8z7Djz3VJtq3sXJfkniTXJ9l7WPvZw7bbJsl57eX285JsM2zZmUn+Pcmv2/38JMnqCzj/ofjfMyz+PZK8OMnVSe5I8v5h62+V5DdJ7mrXPSLJUu2yX7arXdKe76uH7f9fk/wJ+PpQW7vNeu0xNmvn10pyW5Idx/DefSPJO9vX09uf41va+ae0+808x/smzR8Pp7UxvmfYLvdO8sf2+B8YdpzR3v/5KnFD1dskBwJ7A+9pj3XaAs6jkhyU5Jokdyb5UtJ8mySZluSwJDe0789x7e/s8N+dA5L8Efj5sLb9k9zY7u+gJFsm+V37vh0x7NjrJfl5ktvb8/5WkpUXEOcjv7vt+37vsGl2kg+3y96b5Nr2d++KJC9v2zcEvgo8p93mrrZ9rupukjclmdm+f6cmWWssPytp4ouVS2mKeA6wDHDyKOt8ANga2BR4JrAVcNiw5Y8HVgKmAwcAX0qySlV9iKYaemJVrVBVR48WSJLlgS8Au1bVisA2wMUjrLcq8IN23dWAzwA/SLLasNVeC+wPrAEsBbxrlEM/nuZnMJ0mGT4KeB2wObA98MEk67brzgH+BVid5mf3AuAtAFW1Q7vOM9vzPXHY/lelqeIeOPzAVXUt8K/At5IsB3wdOLaqzhwl3iFnATu2r58LXNf+C7AD8KuqqnmO93rgj8BL2xg/NWzxdsAG7Tl9sE2GYOHv/4iq6kjgW8Cn2mO9dJTVdwO2bPf/KuBFbft+7fQ8YF1gBeCIebZ9LrDhsG0Ang2sD7yapir/AWAnYGPgVUmGfk4BPg6s1e5jbeDDYzi3g9tzWoHm53YncEq7+Fqa35uVgMOB/0nyhKq6EjgI+E277XxJbJLnt/G8CngCcANwwjyrLehnJU1sQ2OLW7mUBt5qwG0LuWy9N/CRqrq1qv5C84X5+mHL/94u/3tV/RC4lyZJGY+HgU2SLFtVt1TV5SOs8xLgmqr6ZlXNrqrjgd8Dw5OXr1fV1VX1AHASTWK0IH8HPlZVf6f5Il8d+HxV3dMe/3LgGQBVdUFV/bY97h+Ar/GPhG60c/pQVT3YxjOXqjoKuAY4hyah+MC86yzAWcD2SabRJJOfArZtlz23Xb4oDq+qB6rqEuASmuQFFv7+98Inququqvoj8Av+8X7tDXymqq6rqnuB9wF7Ze5L4B9uK+7Df7b/XlV/q6qfAPcBx7fx3wT8CngWQFXNrKoz2vfmLzR/qCzs/XxEkscB/we8raouavf57aq6uaoebv/AuIYmIR+LvYFjqurCqnqwPd/nJFln2DoL+llJ6oDJpTS/24HVM/r9amvRVFCG3NC2PbKPeZLT+2kqTIukqu6jqTQdBNyS5AdJnjaGeIZimj5s/k+LEM/tVTWnfT2UoPx52PIHhrZP8tQk30/ypyR/panMjnjJfZi/VNXfFrLOUcAmwBfbpGKh2qrnvTTJxfbA94Gbk2zA+JLLBf3MFvb+98KiHHtJmnuDh9w4wv7mff8W9H6ukeSEJDe17+f/sPD3k3bbxwDfAf63qk4Y1r5PkovbS/B30byvY9on85xvm1Dfzvh/t6WJxcvi0pTwG+BvwB6jrHMzzSXdIU9s28bjPmC5YfOPH76wqk6vqhfSVPB+T5N0LSyeoZhuGmdMi+IrNHGtX1WPBd5Pc7FnNDXawiQr0Fy6PRr4cHvZf6zOoumRv1RblTsL2AdYhRFuKRhLPCMY7f2f6/1MMtf7OY5jjeXYs5k7WXw0x/h4u/0z2vfzdSz8/RzyReAeht0ikORJNL+zB9M8IWFl4LJh+1xYrHOdb3uryGosnt9tqc+851KaEqrqbpr7DL+UpiPLckkek2TXJEP34x0PHJbkcWk6xnyQpsIzHhcDOyR5Ytsx431DC5KsmeRl7RfqgzRVuTkj7OOHwFPTPD5pySSvBjaiqdz124rAX4F726rqm+dZ/meaewMXxeeBC9rH3PyAptMH8EgnkjNH2fYsmkRmqDPRmcDbgLOHVWPntagxjvb+XwJsnGTTJMsw//2K4/l5zHvsf0ny5DYJH7qHt1dPH1iR5vfsriTTgXePZaMk/0xTHX5tVT08bNHyNAnkX9r19qepXA75MzAjbSewEfwvsH/781ya5nzPaW/BkCa/aRnfNIGZXEojqKrPAIfSVGD+QnOZ8WCae8kAPgqcD/wOuBS4sG0bz7HOAE5s93UBcyeE04B30lRv7qD58n7LCPu4naZTwztpLhm+B9itqm4bT0yL6F00nYXuoalQnTjP8g8D32gvib5qYTtLsjuwC82tANC8D5ul7SVP08Hk16Ps4iyaBGkouTybppL4ywVu0VTrDmtjHK2j05AFvv9VdTXN0wZ+SnNv4dnzbHs0sFF7rP9j0R0DfJPmfK6nqbK/bRz7WZDDgc2Au2kS+++NcbvX0CTNNw/rMf7+qroC+C+aKwJ/Bp7O3O/fz2nu4f1Tkvl+X6vqZ8C/Ad8FbgHWA/Yaz4lJE87Q8I8DVrnMPB0nJWlCS3Ix8II2oZakSWvaY6fX0lu+dVzb/u3nH7igqrbocUg94QN2JU0qVWVPYEmawDqpqyb5fwtZfnLbs3BmmgdCX9xO24yyzdlJ/NKRJEmTxGB26OmkcllVC0wS2+VDozfsCLyrqnZbHHFJkiQtVhP8gejj0VXl8t723x3TDEv3nSS/TzPM2Kg/5SQvbKuYlyY5apQehpIkSRPbAFYuJ0J0zwIOoXlsyrr8YzSN+bRDwR0DvKKqnk7TA/TABa0vSZI0YY136McJXu2cCB16zq2qWfBIL9B1mP/RHUM2pBni7tp2/jiacZvnHVf3EUkOpE1Al19++c032GCkwU0kTSUPL3wVSQPujzf8gdtvu21iZ2mT1ERILocP6zaH0WNa5F+CqjoSOBJg8823qF+fc/6i7kLSgHlotumlNNU9d9uxDm/fZxP8Evd4TLYzugJYP8nQ6BavY9HHCpYkSZoYvCzeraq6P8kBwPeSLAGcw8jjLEuSJE1wGcjKZVePIlqh/fdMmnF/h9oPnme9uZa3bT8BfjLCPrfreaCSJEn9NMGrkOMxqSqXkiRJA2NobPEBM3hnJEmSpM5YuZQkSeqE91xKkiSpl7znUpIkST1j5VKSJEk9M4CVy8FLlyVJktQZK5eSJEldiB16JEmS1EsDeFnc5FKSJKkjMbmUJElSLwSTS0mSJPVK2mnADN5dpJIkSQIgyRJJLkry/Xb+yUnOSXJNkhOTLNW2L93Oz2yXrzNsH+9r269K8qKFHdPkUpIkqRMhGd+0CN4BXDls/pPAZ6tqfeBO4IC2/QDgzqp6CvDZdj2SbATsBWwM7AJ8OckSox3Q5FKSJKkj/Uwuk8wAXgL8dzsf4PnAd9pVvgHs0b7evZ2nXf6Cdv3dgROq6sGquh6YCWw12nFNLiVJkjrS58rl54D3AA+386sBd1XV7HZ+FjC9fT0duBGgXX53u/4j7SNsMyKTS0mSpI48iuRy9STnD5sOnGe/uwG3VtUFw5tHCKEWsmy0bUZkb3FJkqTJ57aq2mKU5dsCL0vyYmAZ4LE0lcyVkyzZVidnADe3688C1gZmJVkSWAm4Y1j7kOHbjMjKpSRJUhfyKKaFqKr3VdWMqlqHpkPOz6tqb+AXwCvb1fYFTmlfn9rO0y7/eVVV275X25v8ycD6wLmjHdvKpSRJUgfCIvf87oV/BU5I8lHgIuDotv1o4JtJZtJULPcCqKrLk5wEXAHMBt5aVXNGO4DJpSRJUkcWR3JZVWcCZ7avr2OE3t5V9TdgzwVs/zHgY2M9nsmlJElSRxz+UZIkST0ziMmlHXokSZLUM1YuJUmSujDGnt+TjcmlJElSRwbxsrjJpSRJUgc6ehRR35lcSpIkdcTkUpIkSb0zeLmlvcUlSZLUO1YuJUmSuhAvi0uSJKmHTC4lSZLUMyaXkiRJ6olBfRSRHXokSZLUM1YuJUmSujJ4hUuTS0mSpE7YW1ySJEm9ZHIpSZKknjG5lCRJUu8MXm5pb3FJkiT1jpVLSZKkjnhZXJIkST2RDOZD1E0uJUmSOmJyKUmSpJ4ZxOTSDj2SJEnqGSuXkiRJXRm8wqXJpSRJUlcG8bK4yaUkSVIXHFtckiRJvRJgAHNLk0tJkqRuDOZzLu0tLkmSpJ6xcilJktSRASxcmlxKkiR1ZRAvi5tcSpIkdSGDWbn0nktNGT85/cc8Y+MN2PhpT+E/P/WJrsOR1Edv/ecDWO+Jj2frzZ8x37IvfPa/WGnZJbj9ttseafvVL89ku2dvxrM3ezovfuHzFmeomsICTJuWcU0TmcmlpoQ5c+ZwyNvfyimn/YiLfncF3z7heK684oquw5LUJ699/b5895Qfztc+68Yb+cXPz2DttZ/4SNtdd93FO99xMMd/+/8458JL+ca3TlycoUoDx+RSU8J5557Leus9hSevuy5LLbUUe756L75/2ildhyWpT7bdbgdWWXXV+drf955D+cjHPjnXfW7fPvF4Xrr7y1n7iU3C+bg11lhscUrJ+KaJzORSU8LNN9/EjBlrPzI/ffoMbrrppg4jkrS4/fD7p7LWWtN5+jOeOVf7tddczV133clLdn4+O2yzJcd/67iOItRUlGRc00Rmhx5NCVU1X9tE/59TUu/cf//9fPqTH+fk7/94vmWzZ8/m4gsv5NQfncHfHniAnXbcli232pqnrP/UDiLVlDIJqpDjYXKpKWH69BnMmnXjI/M33TSLtdZaq8OIJC1O1193LTfccD3bbfUsoPkM2OE5W/DzX/2WtabPYLXVV2f55Zdn+eWXZ5vttufS311icqm+a4Z/HLzs0svimhK22HJLZs68hj9cfz0PPfQQ3z7xBF6y28u6DkvSYrLxJk/n2j/+iUuvuo5Lr7qO6dNn8MvfnM+aj388L3npy/h/vz6b2bNnc//993PBeeeywdM27DpkTQnjuyQ+0RNSk0tNCUsuuSSf/fwRvPQlL2LTp2/IK/Z8FRttvHHXYUnqkzfs81peuOO2XHP1VWy43hM57tijF7juBk/bkJ1e+CK22XJTnr/91uyz3wFstPEmizFaabBkpHvRBtXmm29Rvz7n/K7DkNSxh2Y/3HUIkjr23G234qILzu+0BLjcWhvUUw/88ri2veTwnS6oqi16HFJPeM+lJElSRyb6Je7xMLmUJEnqgr3FJUmS1Cv2FpckSZIWwsqlJElSRwawcGlyKUmS1JVBvCxucilJktSRAcwtTS4lSZI6ESuXkiRJ6pGmt3jXUfSevcUlSZLUM1YuJUmSOhEvi0uSJKl3BjC3NLmUJEnqipVLSZIk9caAji1uhx5JkqQBk2SZJOcmuSTJ5UkOb9ufnOScJNckOTHJUm370u38zHb5OsP29b62/aokL1rYsU0uJUmSOtA8iijjmsbgQeD5VfVMYFNglyRbA58EPltV6wN3Age06x8A3FlVTwE+265Hko2AvYCNgV2ALydZYrQDm1xKkiR1pF/JZTXubWcf004FPB/4Ttv+DWCP9vXu7Tzt8hekOdDuwAlV9WBVXQ/MBLYa7dgml5IkSR1JxjeNbd9ZIsnFwK3AGcC1wF1VNbtdZRYwvX09HbgRoF1+N7Da8PYRthmRHXokSZI68ih6i6+e5Pxh80dW1ZHDV6iqOcCmSVYGTgY2HGE/NRTKApYtqH2BTC4lSZK68Oh6i99WVVuMZcWquivJmcDWwMpJlmyrkzOAm9vVZgFrA7OSLAmsBNwxrH3I8G1G5GVxSZKkAZPkcW3FkiTLAjsBVwK/AF7ZrrYvcEr7+tR2nnb5z6uq2va92t7kTwbWB84d7dhWLiVJkjqQ/g7/+ATgG23P7mnASVX1/SRXACck+ShwEXB0u/7RwDeTzKSpWO4FUFWXJzkJuAKYDby1vdy+QCaXkiRJHelXbllVvwOeNUL7dYzQ27uq/gbsuYB9fQz42FiPbXIpSZLUkWkDOESPyaUkSVJHBjC3tEOPJEmSesfKpSRJUgeaB6IPXunS5FKSJKkj0wYvtzS5lCRJ6oqVS0mSJPXMAOaWJpeSJEldCM2D1AeNvcUlSZLUM1YuJUmSOmKHHkmSJPVG+jq2eGdMLiVJkjoygLmlyaUkSVIXwmCOLW6HHkmSJPXMAiuXSR472oZV9dfehyNJkjR1DGDhctTL4pcDBXM9gGlovoAn9jEuSZKkgTelOvRU1dqLMxBJkqSpJBnMyuWY7rlMsleS97evZyTZvL9hSZIkDb5pybimiWyhyWWSI4DnAa9vm+4HvtrPoCRJkqaCjHOayMbyKKJtqmqzJBcBVNUdSZbqc1ySJEmahMaSXP49yTSaTjwkWQ14uK9RSZIkTQFTqkPPMF8Cvgs8LsnhwKuAw/salSRJ0oBrHqLedRS9t9DksqqOS3IBsFPbtGdVXdbfsCRJkgbcFB9bfAng7zSXxh3VR5IkqQcGMLccU2/xDwDHA2sBM4D/TfK+fgcmSZKkyWcslcvXAZtX1f0AST4GXAB8vJ+BSZIkDbqpeln8hnnWWxK4rj/hSJIkTQ1TrkNPks/S3GN5P3B5ktPb+Z2BsxdPeJIkSYNrqlUuh3qEXw78YFj7b/sXjiRJ0tQxeKnlKMllVR29OAORJEmaShIm/Djh47HQey6TrAd8DNgIWGaovaqe2se4JEmSNAmN5ZmVxwJfp6nc7gqcBJzQx5gkSZKmhGR800Q2luRyuao6HaCqrq2qw4Dn9TcsSZKkwZd2lJ5FnSaysTyK6ME0Z3FtkoOAm4A1+huWJEnS4JvgeeK4jCW5/BdgBeDtNPdergS8oZ9BSZIkDbqQqdmhp6rOaV/eA7y+v+FIkiRpMhvtIeon0zw0fURV9U99iUiSJGkqmASdc8ZjtMrlEYstCklajNZ8ztu7DkFSxx686o9dhwBMsRF6qupnizMQSZKkqWYsj+2ZbMbSoUeSJEk9FqZY5VKSJEn9NW3wcsuxV2OTLN3PQCRJkjT5LTS5TLJVkkuBa9r5Zyb5Yt8jkyRJGnDTMr5pIhtL5fILwG7A7QBVdQkO/yhJkvSoNOOET83hH6dV1Q3znMicPsUjSZI0ZUz0KuR4jCW5vDHJVkAlWQJ4G3B1f8OSJEkafBO8CDkuY7ks/mbgUOCJwJ+Brds2SZIkaS5jGVv8VmCvxRCLJEnSlBFg2gCWLheaXCY5ihHGGK+qA/sSkSRJ0hQxVUfo+emw18sALwdu7E84kiRJU8cAFi7HdFn8xOHzSb4JnNG3iCRJkqaAJFPzsvgIngw8qdeBSJIkTTUDmFuO6Z7LO/nHPZfTgDuA9/YzKEmSJE1OoyaXaZ6c/kzgprbp4aqar3OPJEmSFt2Ue4h6VVWSk6tq88UVkCRJ0lQwqI8iGksP+HOTbNb3SCRJkqaYZnzxRZ8msgVWLpMsWVWzge2ANyW5FriPJtGuqjLhlCRJGq8M5mXx0SqX57b/7gFsALwY2BN4ZfuvJEmSJqAkayf5RZIrk1ye5B1t+6pJzkhyTfvvKm17knwhycwkvxt+1TrJvu361yTZd2HHHu2eywBU1bWP8vwkSZI0gtC30uVs4J1VdWGSFYELkpwB7Af8rKo+keS9NE8A+ldgV2D9dno28BXg2UlWBT4EbEHz9KALkpxaVXcu6MCjJZePS3LoghZW1WcW5QwlSZL0D02Hnv7su6puAW5pX9+T5EpgOrA7sGO72jeAM2mSy92B49qnAv02ycpJntCue0ZV3QHQJqi7AMcv6NijJZdLACtA/1JqSZKkqWxx3HOZZB3gWcA5wJpt4klV3ZJkjXa16cw9vPestm1B7Qs0WnJ5S1V9ZFGClyRJ0thl/F2/V09y/rD5I6vqyBH2vwLwXeCQqvrrKMcbaUGN0r5AC73nUpIkSb33KC+L31ZVW4y6/+QxNInlt6rqe23zn5M8oa1aPgG4tW2fBaw9bPMZwM1t+47ztJ852nFH6y3+gtE2lCRJ0sTUjrJ4NHDlPP1kTgWGenzvC5wyrH2fttf41sDd7eXz04Gdk6zS9izfuW1boAVWLodu3JQkSVIf9PeB6NsCrwcuTXJx2/Z+4BPASUkOAP7IPx4v+UOax07OBO4H9ocmH0zy78B57XofWViOOOrwj5IkSeqffg3/WFVns+BbHOe7Ot32En/rAvZ1DHDMWI9tcilJktSBfj6KqEsml5IkSR2Z6OOEj8doHXokSZKkRWLlUpIkqRNh2gA++dHkUpIkqQNhMC+Lm1xKkiR1IXbokSRJUg/161FEXTK5lCRJ6sCgXha3t7gkSZJ6xsqlJElSR7wsLkmSpJ4ZwNzS5FKSJKkLYTDvTzS5lCRJ6kIgA1i6NLmUJEnqyOClloNZjZUkSVJHrFxKkiR1INhbXJIkST00eKmlyaUkSVJnBrBwaXIpSZLUjQxkb3E79EiSJKlnrFxKkiR1wIeoS5IkqacG8bK4yaUkSVJHBi+1NLmUJEnqhsM/SpIkqVcG9Z7LQTwnSZIkdcTKpSRJUke8LC5JkqSeGbzU0uRSkiSpMwNYuDS5lCRJ6kLToWfwsks79EiSJKlnrFxqyvjJ6T/mXYe+gzlz5rDfG97Iu9/z3q5DktRj06aFX3/rPdx869284h1f5adHH8IKyy8DwBqrrsj5l/2BVx16FHvtugWH7vdCAO574EHe/h8ncunVNy1wP1K/eFlcmqTmzJnDIW9/Kz/40RlMnzGD7bbekt12exkbbrRR16FJ6qGDX/s8rrr+z6zYJpQ7HfC5R5Yd/+k3ctqZvwPgDzffzs5v/Bx33fMAO2+7EV867DXssM+nF7gfqT9CvCwuTU7nnXsu6633FJ687rostdRS7Pnqvfj+aad0HZakHpq+xsrsst3GfP3k/zffshWWW5rnbvlUTvtFk1z+9pLrueueBwA493fXM33Nlce0H6nXkvFNE5nJpaaEm2++iRkz1n5kfvr0Gdx0002jbCFpsvnPd7+CD3z+/3j44Zpv2cue/0zOPPcq7rnvb/Mt22+PbTj911eMaT9SLw116BnPNJGZXGpKqJr/S2IQH1wrTVW7br8Jt95xDxddeeOIy1+1y+ac9OML5mvfYYv12XeP53DY508Z036knhpn1XKif315z6WmhOnTZzBr1j++LG66aRZrrbVWhxFJ6qXnbLouuz336eyy3cYsvdRjeOzyy3DMR/fhDYcdx6orLc8WG6/Dqw89aq5tNll/Lb7ywdey+8Ff4Y6771vofiSNjcmlpoQtttySmTOv4Q/XX89a06fz7RNP4Nhv/m/XYUnqkQ9+8VQ++MVTAdh+8/U5ZJ8XPJIQ/tMLn8WPfnUZDz40+5H11378Kpzw6TdxwL8dx8w/3jqm/Uj9MNGrkONhcqkpYckll+Sznz+Cl77kRcyZM4d993sDG228cddhSVoM9nzR5nz66z+Zq+19B+7Kqisvz+fe92oAZs95mO32/lQX4WmKG8Te4hnpXrRBtfnmW9Svzzm/6zAkdWyVLQ/uOgRJHXvwqpN4+P5bO83sNthk0/ryd346rm132vBxF1TVFj0OqSesXEqSJHVkECuX9haXJElSz1i5lCRJ6ogdeiRJktQzg3hZ3ORSkiSpAwGmDV5uaXIpSZLUjVi5lCRJUo9MgqEcx8Pe4pIkSeoZK5eSJEkdGcDCpcmlJElSF5oOPYOXXppcSpIkdWTwUkuTS0mSpO4MYHZphx5JkiT1jJVLSZKkjvicS0mSJPXMAPbnMbmUJEnqygDmliaXkiRJnRnA7NLkUpIkqQNhMO+5tLe4JEnSgElyTJJbk1w2rG3VJGckuab9d5W2PUm+kGRmkt8l2WzYNvu261+TZN+xHNvkUpIkqQtpOvSMZxqDY4Fd5ml7L/Czqlof+Fk7D7ArsH47HQh8BZpkFPgQ8GxgK+BDQwnpaEwuJUmSOpJxTgtTVb8E7pineXfgG+3rbwB7DGs/rhq/BVZO8gTgRcAZVXVHVd0JnMH8Cet8vOdSkiSpK4v3lss1q+oWgKq6Jckabft04MZh681q2xbUPiqTS0mSpE7k0XToWT3J+cPmj6yqI8cdyPxqlPZRmVxKkiRNPrdV1RaLuM2fkzyhrVo+Abi1bZ8FrD1svRnAzW37jvO0n7mwg3jPpSRJUkf62KFnJKcCQz2+9wVOGda+T9trfGvg7vby+enAzklWaTvy7Ny2jcrKpSRJUgfG2jlnXPtOjqepOq6eZBZNr+9PACclOT+XACQAAAzTSURBVAD4I7Bnu/oPgRcDM4H7gf0BquqOJP8OnNeu95GqmreT0HxMLiVJkrrSp+yyql6zgEUvGGHdAt66gP0cAxyzKMc2uZQkSerIII7QY3IpSZLUkUdx/+SEZYceSZIk9YyVS0mSpI4MYOHS5FKSJKkT/ewu3iGTS0mSpI7YoUeSJEk9EezQI0mSJI3KyqUkSVJHBrBwaXIpSZLUmQHMLk0uJUmSOmKHHkmSJPXMIHboMbmUJEnqyADmlvYWlyRJUu9YuZQkSerKAJYuTS4lSZI60Iz+OHjZpcmlJElSF2KHHkmSJPXQAOaWduiRJElS71i5lCRJ6soAli5NLiVJkjoRO/RIkiSpd+zQI0mSpJ4IA3lV3ORSkiSpMwOYXdpbXJIkST1j5VKSJKkjduiRJElSz9ihR5IkST0zgLmlyaUkSVInBnRscTv0SJIkqWcmTHKZZL8ka7WvT05ycZKZSe5uX1+cZJtRtj87yaaLL2JJkqRHK+OcJq6JdFl8P+Ay4OaqejlAkh2Bd1XVbh3GJUmS1HPBy+KLLMmhSS5rp0OSrJPksmHL35Xkw0leCWwBfKutUC47yj5f2K5zaZKjkizVz3OQJEnql8GrW/YxuUyyObA/8Gxga+BNwCojrVtV3wHOB/auqk2r6oEF7HM54BjgFVX1dGA54MA+hC9JktR3yfimiayfl8W3A06uqvsAknwP2P5R7nND4JqquradPw44ADhiQRskOZB/JKD3LvuYXPUoY9DktzpwW9dBSOqUnwN6UtcBgA9RX1Qj/bRWZu5q6TI92OeoqupI4MhF3U6DK8n5VbVF13FI6o6fA1L/9POey18CeyRZLsnywMuBHwFrJFktydLA8I469wArLmSfVwDrJ1m3nX8dcFaP45YkSVo8BvCmy75VLqvqwiTHAue2Tf9dVecl+QhwDnA98PthmxwLfDXJA8BzRrrvsqruT3IA8L0kS7T7Oapf5yBJktRPEzxPHJdUVdcxSItVkgPb2yUkTVF+Dmgi2HSzzesnZ/12XNuu+dilLpiot3ZMpOdcSouFXyiS/BzQRDGIHXomzAg9kiRJmvxMLjVltJ3IJE1hSR7TdQzSXAawQ4/JpaaEJJsAX0syIZ5rJmnxS7Ip8Jkkj+86FmnIAOaWJpeaGqrqMmBZ4LAkT+w6HkmdmAmsB7wnyRpdByPBYI7QY3KpgZbGEgBV9WrgMcBHkqzdbWSSFpehz4Gquhd4Nc3ILP9mgqnuZdz/TWQmlxpYSaZVY06S1QCqaj/gPuA/rGBKg2+ez4HHVtU9NANwrAp8yARTXQpWLqVJpaoeBkjyJuDYJF9Ksm9VvRW4C/hwknU6DFFSnw37HDgAOCbJp4GdgX2AlYAPeA+m1Fsmlxo4SZ46VI1I8grg7cBHgPOA7ZK8p6reBiwPvCuJz3uVBkySdZOs0r5+DXAI8CXgWmBP4A3ttAFwyNDtM5IePZNLDZQkLwa+yz/GqV8J+FpVnQecBPw3sGmSldt7MP+jqmZ3E62kfmg/B74NPLZtWg34VFX9AvgmzbDBO7TL9gSOqKo5iz1QCS+LSxNakhcBRwCvq6pr2+a7gYOTPK2q7q+qc2i+cJ4KUFU3dxOtpH5oPwc+B+xfVTe0zX+nuUrxhLZTzzk091yuX1X3VNWsjsKVBrJDj5cDNRDaL5Sv0CSTM4BLAKrqu+19lZ9N8nFgDWB14KZuIpXUL+3nwNdoPgdWGLboKODxwFeSvB94GrAycMdiD1IabhJUIcfD5FKTXpJtgU8CBwH3A/+aZLWqOq5d5evAQ8B7gQeAf64qk0tpgCTZgeZz4M1t0weSfK2qTq2qh5N8FXgL8BmggLdU1S0dhSsNNJNLDYLbgTdU1YVJVgSOBt6QJFX1jaq6A/hikiOBh6vq751GK6kf7gfeVFXnJVkdOB44sP0cOKWq/kzz6KFlaT4HHuw0WonJMdrOeJhcatJqn1/3cFX9vp1PVd2T5Kc0lYkDksypqv8B8MtEGjxtIrl8VZ2f5Hntw9JvS3IqzefAm9rPge8DVNUDnQYszWsAs0uTS01KbSI59Py6F9P87/lb4PaqujfJz2i+WN6Z5O9VdWKH4Urqn/VpKpR3AbsC29J8Dvw1yWnAwzS3ysyuqh93Gag0koneOWc8TC41qSQJMDyx3B94J82N+Rcl+V5VndUmmL8AZgOXdhexpH5o/8CsqvpNkr1o7rl+fVXdPmzZX5P8mOZz4PJuI5ZGZoceqXtLDD2XMslLgH8Cnk7zu/whYLf2e+WX7SXyH1VVdRivpD4Y+v86yTOB04GZwD5Jbgd+Ccxp17szyXeH/iCVJpoBzC19zqUmjySPA76XxpLAFsBzga3aTjqfBx4E9mp7kGNiKQ2uJOsCbwI2qqovAqcB76MZKOGQJJ+EfwwBKU01SXZJclWSmUneu7iOa3KpSaOq/gLsBewErFBVhwNfBN6b5Bnt8i8CfwKu6S5SSYtDVV0HnAnMSPKOqjoK+A7wL8BrgRM6DE8am4xzWthumyFNv0RzL/JGwGuSbNTz+EdgcqlJparupxkT/LIkKwGH0YwZ/qEkm7WPG/loVd3aZZyS+ifJq5L8C0BVfYcmwdwgyUE0jyI7CNi5qi7qLkppbPo4Qs9WwMyquq6qHqL5Y2v3vp5My+RSk05V/R/wRuB8mqEcPw5cSdMzfGmaXuKSBkSSacNeLwvMAvZN8kZ45DPhL8CBwD8D91XVXV3EKi2K0NexxacDNw6bn9W29Z0dejQpVdWPk7yD5vFDz6mqw9pReXyWpTRghj0d4iBga5ovyf+iuSVmTlV9naY3+JrAd73XWpPFhRdecPqyj8nq49x8mSTnD5s/sqqOHDY/Ugq6WP7fMLnUpFVVP0yyFPCzJFtU1e1dxySpP5K8AjgYeB2wP81Y4T8GPtU+6/YZwEvbW2OkSaGqdunj7mcBaw+bnwHc3MfjPSL+gafJLskKVXVv13FI6p8k7wceqqpPt39U7gdsAnwB2AC4vKr+0F2E0sTSPlXlauAFwE00/RNeW1V9f+ar91xq0jOxlKaEK4Dtk2xUVQ+1l/+eQfPEsR+YWEpza58JfTDNc2CvBE5aHIklWLmUJE0CSVYG3kVzH9mZwLLAB4Fd28eQSZogTC4lSZNCkrWAVwAvBe4FDq+qS7qNStK8TC4lSZNKkuVovr/u6zoWSfMzuZQkSVLP2KFHkiRJPWNyKUmSpJ4xuZQkSVLPmFxKkiSpZ0wuJUmS1DMml5L6KsmcJBcnuSzJt9vHyIx3Xzsm+X77+mVJ3jvKuisnecs4jvHhJO8aa/s86xyb5JWLcKx1kly2qDFK0kRmcimp3x6oqk2rahPgIeCg4QvTWOTPoqo6tao+McoqKwOLnFxKkh4dk0tJi9OvgKe0Fbsrk3wZuBBYO8nOSX6T5MK2wrkCQJJdkvw+ydnAPw3tKMl+SY5oX6+Z5OQkl7TTNsAngPXaqul/tuu9O8l5SX6X5PBh+/pAkquS/BTYYGEnkeRN7X4uSfLdeaqxOyX5VZKrk+zWrr9Ekv8cdux/frQ/SEmaqEwuJS0WSZYEdgUubZs2AI6rqmcB9wGHATtV1WbA+cChSZYBjqIZ7m974PEL2P0XgLOq6pnAZsDlwHuBa9uq6buT7AysD2wFbApsnmSHJJsDewHPokletxzD6XyvqrZsj3clcMCwZesAzwVeAny1PYcDgLurast2/29K8uQxHEeSJp0luw5A0sBbNsnF7etfAUcDawE3VNVv2/atgY2AXycBWAr4DfA04PqqugYgyf8AB45wjOcD+wBU1Rzg7iSrzLPOzu10UTu/Ak2yuSJwclXd3x7j1DGc0yZJPkpz6X0F4PRhy06qqoeBa5Jc157DzsAzht2PuVJ77KvHcCxJmlRMLiX12wNVtenwhjaBHD4udIAzquo186y3KdCrMWoDfLyqvjbPMQ4ZxzGOBfaoqkuS7AfsOGzZvPuq9thvq6rhSShJ1lnE40rShOdlcUkTwW+BbZM8BSDJckmeCvweeHKS9dr1XrOA7X8GvLnddokkjwXuoalKDjkdeMOwezmnJ1kD+CXw8iTLJlmR5hL8wqwI3JLkMcDe8yzbM8m0NuZ1gavaY7+5XZ8kT02y/BiOI0mTjpVLSZ2rqr+0FcDjkyzdNh9WVVcnORD4QZLbgLOBTUbYxTuAI5McAMwB3lxVv0ny6/ZRPz9q77vcEPhNWzm9F3hdVV2Y5ETgYuAGmkv3C/NvwDnt+pcydxJ7FXAWsCZwUFX9Lcl/09yLeWGag/8F2GNsPx1JmlxS1asrTpIkSZrqvCwuSZKknjG5lCRJUs+YXEqSJKlnTC4lSZLUMyaXkiRJ6hmTS0mSJPWMyaUkSZJ6xuRSkiRJPfP/AbkGMpIHGhvGAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 720x360 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "\n",
    "from sklearn.utils.multiclass import unique_labels\n",
    "def plot_confusion_matrix(y_true, y_pred, classes,\n",
    "                          normalize=False,\n",
    "                          title=None,\n",
    "                          cmap=plt.cm.Blues):\n",
    "    \"\"\"\n",
    "    This function prints and plots the confusion matrix.\n",
    "    Normalization can be applied by setting `normalize=True`.\n",
    "    \"\"\"\n",
    "    if not title:\n",
    "        if normalize:\n",
    "            title = 'Normalized confusion matrix'\n",
    "        else:\n",
    "            title = 'Confusion matrix, without normalization'\n",
    "\n",
    "    # Compute confusion matrix\n",
    "    cm = confusion_matrix(y_true, y_pred)\n",
    "    # Only use the labels that appear in the data\n",
    "#     classes = classes[unique_labels(y_true, y_pred)]\n",
    "    if normalize:\n",
    "        cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]\n",
    "        print(\"Normalized confusion matrix\")\n",
    "    else:\n",
    "        print('Confusion matrix, without normalization')\n",
    "\n",
    "    print(cm)\n",
    "\n",
    "    fig, ax = plt.subplots(figsize=(10,5))\n",
    "    im = ax.imshow(cm, interpolation='nearest', cmap=cmap)\n",
    "    ax.figure.colorbar(im, ax=ax)\n",
    "    # We want to show all ticks...\n",
    "    ax.set(xticks=np.arange(cm.shape[1]),\n",
    "           yticks=np.arange(cm.shape[0]),\n",
    "           # ... and label them with the respective list entries\n",
    "           xticklabels=classes, yticklabels=classes,\n",
    "           title=title,\n",
    "           ylabel='True label',\n",
    "           xlabel='Predicted label')\n",
    "\n",
    "    # Rotate the tick labels and set their alignment.\n",
    "    plt.setp(ax.get_xticklabels(), rotation=45, ha=\"right\",\n",
    "             rotation_mode=\"anchor\")\n",
    "\n",
    "    # Loop over data dimensions and create text annotations.\n",
    "    fmt = '.2f' if normalize else 'd'\n",
    "    thresh = cm.max() / 2.\n",
    "    for i in range(cm.shape[0]):\n",
    "        for j in range(cm.shape[1]):\n",
    "            ax.text(j, 0.5*i+0.25, format(cm[i, j], fmt),\n",
    "                    ha=\"center\", va=\"center\",\n",
    "                    color=\"white\" if cm[i, j] > thresh else \"black\")\n",
    "    fig.tight_layout()\n",
    "    return ax\n",
    "plot_confusion_matrix(y_test, y_pred, classes=['outTol', 'inTol'],\n",
    "                      title='Confusion matrix, without normalization')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is super obvious that without rebalancing, the prediction model is biased and keep saying in tolerance. I'll find some time to fix this issue."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}