notebooks/01_simple_forward_pass.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Simple forward pass\n",
    "\n",
    "In this notebook you will do a simple forward pass for a neural network with and without hidden layers. You will use the sigmoid function to calculate the probability if a banknote is fake or not.\n",
    "\n",
    "**Dataset:** You work with a banknote data set and classification task. We have 2 features.\n",
    "\n",
    "* x1: skewness of wavelet transformed image  \n",
    "* x2: entropy of wavelet transformed image  \n",
    "\n",
    "Don't bother too much how these features exactely came from.\n",
    "\n",
    "**The goal is to classify each banknote to either \"real\" (Y=0) or \"fake\" (Y=1).**\n",
    "\n",
    "\n",
    "**Content:**\n",
    "* calculate the forward pass of the neural network without hidden layer by hand, with matrix multiplication and keras\n",
    "* visualize the learned decision boundary in a 2D plot\n",
    "* calculate the forward pass of the neural network with one hidden layer (8 nodes) with matrix multiplication and keras\n",
    "* visualize the learned decision boundary in a 2D plot\n",
    "* compare the decision boundaries of the two models"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import math\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "from tensorflow.keras.models import Sequential\n",
    "from tensorflow.keras.layers import Dense \n",
    "from tensorflow.keras.utils import to_categorical \n",
    "from tensorflow.keras import optimizers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# definition of the sigmoid function\n",
    "def sigmoid(z):\n",
    "    return (1 / (1 + np.exp(-z)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# we just assume x1 and x2\n",
    "x1 = 1\n",
    "x2 = 2.2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# we just assume w1 and w2 and b1\n",
    "w1 = 0.3\n",
    "w2 = 0.1\n",
    "b  = 1 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Forwad pass by hand"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.52"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "(x1*w1+x2*w2)+b ## output before the activation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.8205384805926733"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sigmoid((x1*w1+x2*w2)+b) ## output after the sigmoid activation\n",
    "                         ## probability for the banknote to be fake"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Forwad pass with matrix multiplication"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "X=np.array([[x1,x2]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "W=np.array([[w1],[w2]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(1, 2)\n",
      "(2, 1)\n"
     ]
    }
   ],
   "source": [
    "print(X.shape)\n",
    "print(W.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1.52]])"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "np.matmul(X,W)+b  ## output before the activation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.82053848]])"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sigmoid(np.matmul(X,W)+b) ## output after the sigmoid activation\n",
    "                          ## probability for the banknote to be fake"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "## funtion to return the probaiblity output after the matrix multiplication\n",
    "def predict_no_hidden(X):\n",
    "    return sigmoid(np.matmul(X,W)+b)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "### funtion to plot the decision boundary \n",
    "def plotModel(predict, t):\n",
    "    # define a grid for the 2D feature space\n",
    "    # predict at each grid point the probability for class 1\n",
    "\n",
    "    x1list = np.linspace(-10, 10, 10) # Define 100 points on the x-axis\n",
    "    x2list = np.linspace(-10, 10, 10) # Define 100 points on the x-axis\n",
    "    X1_grid, X2_grid = np.meshgrid(x1list, x2list)\n",
    "\n",
    "    # model.predict for respective value x1 and x2 \n",
    "    p = np.array([predict(np.reshape(np.array([l1,l2]),(1,2))) for l1,l2 in zip(np.ravel(X1_grid), np.ravel(X2_grid))])\n",
    "    print(p.shape)\n",
    "    if len(p.shape) == 3 and p.shape[2]==2:\n",
    "        p = p[:,:,1] # pick p for class 1 if there are more than 2 classes\n",
    "    p = np.reshape(p,X1_grid.shape)\n",
    "\n",
    "    # visualize the predicted probabilities in the 2D feature space\n",
    "    # once without and once with the data points used for fitting\n",
    "    plt.figure(figsize=(16,4))\n",
    "    plt.subplot(1,2,(1))\n",
    "    cp = plt.contourf(X1_grid, X2_grid, p,cmap='RdBu_r')\n",
    "    plt.colorbar(cp)\n",
    "    plt.title(t)\n",
    "    plt.xlabel('x1')\n",
    "    plt.ylabel('x2')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercise \n",
    "Play around with the values for x1 and x2 and check if the position at the decision boundary\n",
    "matches the predicted probability\n",
    "How does the decision boundary look? \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(100, 1, 1)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([0.82053848])"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAcIAAAEWCAYAAAD1t5d8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3de7hcZXn+8e+dhAByDESRUzdnSkDBEAGLpCAIIRaiViG0KlZsfhSoWrWKBZFGaVGrWCtIQ6WoqBAP0KBRDhIakQYS0gAJBwmRSEKEAhGIICHJ8/tjvTusPXtm9t7ZM7PWzNyf65ora9b7rlnPrD2ZZ553nRQRmJmZdasRRQdgZmZWJCdCMzPrak6EZmbW1ZwIzcysqzkRmplZV3MiNDOzruZE2MYk7S9pkaTnJX2o6HjKTtISSUeXdf2SbpP0wRaGlF/3VZI+V6d9jaS9arS9X9LtdZZtyvuSdKGkqxv9utZ9RhUdgA3LJ4A5EXFI0YGUjaSrgBURcX7vvIg4sLiI+q5f0oXAPhHxnmavNyXfqyNit019jYjYunERmZWLK8L21gMsKTqIVpPkH3BWCH/2OpMTYZuSdCtwDPC1NGy1n6QtJX1J0nJJz0q6Pc3bQ1JIOl3SbyQ9Jem83GtdKGmmpG+lYdYlkibUWK8kXSLpSUnPSbpP0kGpbXNJ/5LW8YSkyyVtmdqOlrRC0j+k9T8q6S9zr/s2Sf+bXvOxVDH1tvXGf4ak3wC3pvnfl/Tb9F7nSjowzZ8G/CXwibRtbkjzH5V0XC7Wr0h6PD2+Imnzilg/lt7nKkl/VWN7HCPpvtzzmyXNzz3/haS359cvaRLwD8CpKb57ci/ZI+mX6e9wk6Sxudc6Of1tfpeGGw/ItYWkfXLPr5L0OUlbAT8FdknrWiNpl2rvBRgj6Sdp3XdK2rva60vaUdKs9Le6C9g7/yKS3irpwfR3+RqgivYPSHpA0mpJN0rqqVjPmZIeTu/zUkl9lq+lzufhjenzODLX9529213SCEnnSnpE0tPp/8IOqa3qZ886TET40aYP4Dbgg7nnl6Z5uwIjgT8BNgf2AAK4AtgSOBh4CTggLXch8Adgclrun4F5NdZ5AnA3sD3ZF9wBwM6p7RJgFrADsA1wA/DPqe1oYB3w5RTTnwK/B/bPtb+O7MfZ64EngLentt74vwVsBWyZ5n8grWdz4CvAolycVwGfq4j9UeC4ND0dmAe8Bng1cAfw2YpYpwObpe3yAjCmyvbYMm27sanvE8DKFNeWwIvAjlXWfyHZcGXl3/MRYL+07G3Axaltv7S93prW8wlgKTA6tQfZUGu/95/ez4oBPktXAU8Dh5HtMvkOcE2ufePrA9cAM9Pf4qD0fm9PbWOB54F3pTj/Lm3LD6b2KSnuA9J6zgfuqFjPj8k+X38E/B8wqUbMfbbhAJ+H+4ETc8+vAz6Wpj+cPgu7pWX/Hfhevc+eH531KDwAP4bxx8slQrIE8iJwcJV+vf+Zd8vNuwuYmqYvBG7JtY0DXqyxzrcAvwKOAEbk5it9Ue+dm/cm4Ndp+uj0hbhVrn0m8Oka6/kKcElF/HvV2Rbbpz7bpedXUT8RPgJMzrWdADyai/VFYFSu/UngiBrr/gXwzrRNbkrvaxJZxX5vjfX3+RLP/T3Pzz0/C/hZmv40MDPXNoIsAR2dnjciEf5H7vlk4MHc8wD2Ifuh9DLwx7m2f+KVRPg+cj+i0udiBa98Tn8KnFHxPl4AenLreXPFZ+TcGjH324Z1Pg+fBL6TpndI6+z9AfcAcGxu2Z3TexzFID57frT/w+PdnWMssAXZF3wtv81NvwBsXadtC0mjImJd/gUi4tY03HUp2TDej4CPp3W/Crg7N5Ilsi/OXqsj4ve558uBXQAkHQ5cTFZhjCb7Zf79ivgf2/jC2TDXRcC7ySq6DalpLPBsjfeft0taf79Ykqcr3nvl9sr7b1KySdOrySrel9Lzoaj1N+oTb0RskPQYWfXfKPU+H71eTZYgHsvNy2/HXfJtEREpzl49wL9K+lJunsjeR+/rDCaOPgbxebgaeCANFZ8C/CIiVuViuk7ShtxLrgd2yj3PvwfrMN5H2DmeIhui23ugjsMVEV+NiEPJKsf9gL9P638RODAitk+P7aLv0YZj0hdRrz8CHk/T3yUbVt09IrYDLqdi3xLZL/Nef0E2zHYcsB3ZL3dyywx0W5XHyb4Aq8UyVL2JcGKa/m+yRPin1E6EQ73tS594036z3cmqQsgSxqty/V87jHXV839klf3uuXl/lJtelW/LxdnrMeD/5T4j20fElhFxxzDjqvt5iIiVwP+QVe7vBb5dEdOJFTFtkZbp5dv0dDAnwg4RERuAK4EvS9pF0khJb+o9AKRR0oEHh0vajGwo9A/AhrT+K4BLJL0m9d1V0gkVL/GPkkZLOgr4M16p+rYBnomIP0g6jOyLrZ5tyCqup8kSwD9VtD8BVD3vLfkecL6kV6cDUi4gqxo2xR3A/mT71+6KiCVkSetwYG6NZZ4A9pA02P+DM4G3STo2bfuPkb3/3gSyCPiL9HefRJaE8+vaUdJ2Q3lT1UTEeuBHwIWSXiVpHHB6rstPgAPTwSijgA/RNylfDnwqdyDLdpLePdy4GPjzANl+vk+Q7Yv+UUVMF/UetJM+E1MaEJO1CSfCzvJx4D5gPvAM8Hka/zfelizhrSYbynoa+GJq+yTZgRDzJD0H3EKWIHr9Ni33ONnBGGdGxIOp7SxguqTnyZLSzAHi+FZa/0qyAyHmVbR/AxiXjjy8vsrynwMWAPeSbbOFad6QpeHehcCSiFibZv8PsDwinqyxWO8PgKclLRzEOh4C3gP8G1n1fRJwUm59H07zfkd2xOz1uWUfJEv8y9L2qHXU6GCdQzZc+VuyfYv/mVvXU2TDkxeTfTb2BX6Za7+O7HN5TfqMLAZOHGY8MPDnAbIDZHqA6yLihdz8fyUbjbgpff7mkf2IsS6hCFf81nxqwEndZsMl6RGyodlbio7FysMVoZl1BUl/Travz+cCWh+FJkJJVyo7YXlxbt4Oyk5Kfjj9O6bGsqenPg9LOr1aHzMzyK53CnwdODvtz7aSq5YfKtol6auSlkq6V9L4XNt6ZddhXiRp1oDrKnJoVNJEYA3wrYjovTrJF8gOmrhY0rlkJzF/smK5Hcj270wg+4V3N3BoRKxu6RswM7OmqJYfKtonA39Lds7r4cC/RsThqW1NDOH6uIVWhBExl+ygjrwpwDfT9DeBt1dZ9ATg5oh4JiW/m8lOYDYzsw5QIz/kTSFLkhER84DtJe28Kesq4wn1O+VOdP0tfU9q7bUrfU9wXUGNE4uVXXdyGsDIzbc8dOudeqp149XbjN7UeBtu29EjB+7UCi8M5rz01lv7TL3/G53n+WdeLDoE60DL17/0VES8upGvedBmW8WaWD/Y9S8hO/2q14yImDGE1dXKA6vILgiygOyc14sjotqR4xuVMRFulK5KMayx27RhZwBs33NAHHXelVX7nTWx3ilnrXdMz7ZFh8CIhTcUHUJVy68e6MyKzjL32qq7SMyG5YOrH1o+cK+hWRPr+fS2ewx2/X+IiKoX92+AnohYqewemrdKui8ial51q4xHjT7RW96mf6udh7WSvler2I1XrrCxSS6bu2w4i1sL9bznlKJDMLPi1cwDvVcFiohlZNfwfUO9FypjIpzFK1eqOB34ryp9bgSOlzQmHVV6fJrXMeYsf67oENgw/qSiQzBg4qn9jhMwsyxXvC8dPXoE8GxErEp5ofeWamOBI8kuslBT0adPfI/sChz7K7v/2xlkV6R4q6SHya4beHHqO0HSfwBExDPAZ8muoDIfmJ7mDYurwvbhqtCss1XLD8ruVXlm6jIbWEZ2NasryK5OBdktvhYou9/kHLJ9hHUTYaH7CCPitBpNx1bpuwD4YO75lWTX1uxYc5Y/V/i+wg3jTyrtvsJuMvHUg7yv0LpKnfzQ2x7A2VXm30F2PdlBK+PQqJVMWYdIXRWaWSM4EVYo2/BoGfYVmpl1MidCGxRXhcXzQTNmzeFEWIWrQjOz7uFEaIPmqrB4rgrNGs+JsAZXhWZm3cGJ0IbEVWHxXBWaNZYTYR2uCs3MOp8ToQ2Zq0Iz6yROhANwVWhl5OFRs8ZxIrRN4qrQzDqFE+EguCq0MnJVaNYYToS2yVwVFs/J0Gz4nAgHyVWhmVlnciK0YXFVWDxXhWbD40Q4BK4Kzcw6jxOhDZurwuK5KjTbdE6EQ1S2qtDqczI0s4GUMhFK2l/SotzjOUkfqehztKRnc30uKCreIpVleLSsVaGZ2UBKmQgj4qGIOCQiDgEOBV4ArqvS9Re9/SJieqvic1XYXlwVmlk9pUyEFY4FHomI5UUHUlauCs3MNl07JMKpwPdqtL1J0j2SfirpwFYG5arQyspVodnQlDoRShoNnAx8v0rzQqAnIg4G/g24vsZrTJO0QNKCtWtWNy/YgrkqrK+bhkfNOoGkSZIekrRU0rlV2nsk/VzSvZJuk7Rbru10SQ+nx+kDravUiRA4EVgYEU9UNkTEcxGxJk3PBjaTNLZKvxkRMSEiJozeekxDg3NVaGXlqtDamaSRwKVkOWAccJqkcRXd/gX4VkS8HpgO/HNadgfgM8DhwGHAZyTV/fIveyI8jRrDopJeK0lp+jCy9/J0C2MrHVeF9bkqNGsbhwFLI2JZRKwFrgGmVPQZB9yapufk2k8Abo6IZyJiNXAzMKneykqbCCVtBbwV+FFu3pmSzkxP3wUslnQP8FVgakREq+N0VWhl5arQSm5s726r9JiWa9sVeCz3fEWal3cP8M40/Q5gG0k7DnLZPkZtSvStEBG/B3asmHd5bvprwNdaHVfZzVn+HMf0bFt0GGwYfxIjFt5QdBj99LznFJZfPbPoMFpm4qkHMffaxUWHYV1imx22ZOKfD/IH2OUPPRURE4axuo8DX5P0fmAusBJYvykvVNqKsJ24KjQza6iVwO6557uleRtFxOMR8c6IeANwXpr3u8EsW8mJsAN5X2F93bav0EOk1obmA/tK2jOdPTAVmJXvIGmspN4c9ingyjR9I3C8pDHpIJnj07yanAgbxFWhmVljRMQ64ByyBPYAMDMilkiaLunk1O1o4CFJvwJ2Ai5Kyz4DfJYsmc4Hpqd5NTkRdihXhfW5KjQrt4iYHRH7RcTeEdGb5C6IiFlp+gcRsW/q88GIeCm37JURsU96/OdA63IibCBXhWZm7ceJsIOVpSosK1eFZgZOhA3nqrC/sg6PmpmBE2HHc1VYn6tCM3MibAJXhf25KjSzsnIibJIyJUNXhfW5KjTrbk6E1jKuCsvDydDsFU6ETeSq0Mys/JwIraXKWhV22/AouCo06+VE2GSuCs3Mys2J0FrOVWF5uCo0cyJsCVeFZmbl5URohXBVWB6uCq3bORG2iKtCM7NyciLsUmVIhq4Ky8NVoXWz0iZCSY9Kuk/SIkkLqrRL0lclLZV0r6TxRcQ5FGWqCs3MLFPaRJgcExGHRMSEKm0nAvumxzTg6y2NrAO4KqzNVaFZ9yh7IqxnCvCtyMwDtpe0c9FBDcRVoZlZuZQ5EQZwk6S7JU2r0r4r8Fju+Yo0rw9J0yQtkLRg7ZrVTQp1aMqUDF0VWp6rQutGZU6Eb46I8WRDoGdLmrgpLxIRMyJiQkRMGL31mMZGaB2tG4dHzbpRaRNhRKxM/z4JXAccVtFlJbB77vluaV5bcFXYl6vC8nBVaN2mlIlQ0laStumdBo4HFld0mwW8Lx09egTwbESsanGo1uFcFZp1vlImQmAn4HZJ9wB3AT+JiJ9JOlPSmanPbGAZsBS4AjirmFA7g6tCMysTSZMkPZROkTu3Svsl6fS6RZJ+Jel3ubb1ubZZA61rVKODb4SIWAYcXGX+5bnpAM5uZVyNdtncZZw1ca+iw7AB9LznFJZfPbPoMFpq4qkHMffaykEYs9aQNBK4FHgr2YGQ8yXNioj7e/tExN/l+v8t8IbcS7wYEYcMdn1lrQitAK4KzawkDgOWRsSyiFgLXEN2ylwtpwHf29SVOREWrEwHzVht3biv0AfNWJON7T21LT3yp8kN6vQ4AEk9wJ7ArbnZW6TXnCfp7QMFUsqhUSvOnOXPcUzPtoXGsGH8SYxYeEOhMZjZ0I3eYYfB/2i8/IdP1bhq2FBNBX4QEetz83oiYqWkvYBbJd0XEY/UegFXhCXgqrA9uCo0a5mhnB43lYph0dzpd8uA2+i7/7AfJ0Lrx/sKzaxg84F9Je0paTRZsut39KekPwbGAP+TmzdG0uZpeixwJHB/5bJ5ToQl4aqwPbgqNGu+iFgHnAPcCDwAzIyIJZKmSzo513UqcE06i6DXAcCCdPrdHODi/NGm1XgfoVXlfYVmVqSImE12vnh+3gUVzy+sstwdwOuGsi5XhCXiqrA9uCo06yxOhFaT9xXW1o3J0KxTORGWjKtCM7PWciK0ulwV1tZtVaGHR61TORGamVlXcyIsobINj7oqrM1VoVn7cyI0M7Ou5kRYUq4K+3NVWA6uCq3TOBGamVlXcyIsMVeF/bkqNLNGcyI0syHz8Kh1ktIlQkm7S5oj6X5JSyR9uEqfoyU9K2lRelxQ7bU6gavC/lwVmlkjlS4RAuuAj0XEOOAI4GxJ46r0+0VEHJIe01sboll13ZQMXRVapyhdIoyIVRGxME0/T3YLjl2LjapYZasKy6CsVWEzrVn7MjMf+DVfnHcfMx/4NWvWvlx0SGYdoXSJME/SHmR3Fr6zSvObJN0j6aeSDqzzGtMkLZC0YO2a1U2KtLuUYXi0zJpRFS5Y9RRHfXs2F/3yHq5Y9Csu+uU9HPXt2SxY9VTD1zUUrgqtE5Q2EUraGvgh8JGIqPzmXQj0RMTBwL8B19d6nYiYERETImLC6K3HNC/gJnNV2F+3VIVr1r7MX8/+Jb9/eR0vrlsPwIvr1vP7l9dtnG9mm66UiVDSZmRJ8DsR8aPK9oh4LiLWpOnZwGaSxg70ui88/0LDY+1Wrgrra2RVOPuRFWzocwPuV2yIYPbSxxq2LrNuVLpEKEnAN4AHIuLLNfq8NvVD0mFk7+Pp1kVZDFeF/XVDVbj82TUbK8FKL65bz/Jn17Q4or48PGrtrnSJEDgSeC/wltzpEZMlnSnpzNTnXcBiSfcAXwWmRtT4yVzh7p8vak7UXchVYX2Nqgp7ttuaLUeNrNq25aiR9Gy3dUPWY9atSpcII+L2iFBEvD53esTsiLg8Ii5Pfb4WEQdGxMERcURE3FF03K3iqrC/Tq8KJ++9GyOyAZB+RkhM3mf3FkfUn6tCa2elS4TWXlwV1teIqnDr0ZtxxeQj2WqzURsrwy1HjWSrzUZtnG9mm64rE2G7D4+6Kuyv06vCCTuP5fb3vY3zjzyYaYfsx/lHHszt73sbE3Ye8BixlnFVaO2qKxOhNZarwvoata9wq81G8e4D9uTjR7yOdx+wpytB62iSJkl6SNJSSefW6HNK7nKc383NP13Sw+lx+kDr6tpE6Kqw83R6VdgOXBVaI0gaCVwKnAiMA06rvNSmpH2BTwFHRsSBwEfS/B2AzwCHA4cBn5FU9yTyrk2E1liuCuvzNUjNhuQwYGlELIuItcA1wJSKPn8NXBoRqwEi4sk0/wTg5oh4JrXdDEyqt7KuToTtXhVaf64KzTrCrkD+ShEr6H/N6f2A/ST9UtI8SZOGsGwf3snQxi6bu4yzJu5VdBgbzVn+HMf0bFt0GGwYfxIjFt5QdBj99LznFJZfPbPoMFpi4qkHMffaxUWHYa32qu2G8mN0rKQFueczImLGENY2CtgXOBrYDZgr6XVDWL7PC5mZmbXaUxExoUbbSiB/guxuaV7eCuDOiHgZ+LWkX5ElxpVkyTG/7G31AunqoVFo/+HRsh00U5Z9hWUdIvW+QrNBmQ/sK2lPSaOBqcCsij7XkxJeutb0fsAy4EbgeElj0kEyx6d5NXV9IjSz5nEytE0REeuAc8gS2APAzIhYImm6pJNTtxuBpyXdD8wB/j4ino6IZ4DPkiXT+cD0NK8mJ0JcFTaaq8L6uqkqNNtU6dKa+0XE3hFxUZp3QUTMStMRER+NiHER8bqIuCa37JURsU96/OdA63IiNLOmclVoZedEmLgqbKyyVIVl5arQrDycCDtI2ZJhGZR1eLTbuCq0MnMizGn3qrBsXBXW56rQrBycCDuMq8L+XBWWg6tCKysnwgquChvLVWF9rgrNiudE2IFcFfbnqrAcXBVaGdVNhJK2lbR3lfmvb15IG9dR915UkjaXdG1qv1PSHs2OyTaNq8L6XBWaFatmIpR0CvAg8MN008M35pqvamZQg7kXFXAGsDoi9gEuAT7fqPV3wvCoq8L+XBWWg6tCK5t6FeE/AIdGxCHAXwHflvSO1KYmxzWYe1FNAb6Zpn8AHCup2XHZJnJVWF+3VYVOhlYm9RLhyIhYBRARdwHHAOdL+hAQTY5rMPeT2tgnXZfuWWDHyheSNE3SAkkLNry0ZtABuCrsTK4KzaxSvUT4fH7/YEqKR5NVYgc2Oa6GiYgZETEhIiaM2HzrosPpaq4KLc9VoZVFvUT4N8CI/L65iHie7Jb3H2xyXIO5F9XGPpJGAdsBTzcyCFeFnamsVWG3DY+alUXNRBgR90TEw8BMSZ9UZkvgy8BZTY5rMPeimgWcnqbfBdwaEc0esrVhclVoea4KrQwGcx7h4WSV1x1kCepx4MhmBjXIe1F9A9hR0lLgo0C/UywawVVhZ3JVaGa9Rg2iz8vAi8CWwBbAryNiQ1OjIrsXFTC7Yt4Fuek/AO9udhzWeHOWP8cxPdsWHYaVxMRTD2LutYuLDsO62GAqwvlkifCNwFFk5/R9v6lRlYyrws7kqtDMYHCJ8Ix0V+CXI2JVREyh//46syHxvkLL875CK9KAiTAiFlSZ9+3mhFNergo7k6tCM/NFt60wrgotz1WhFcWJcAhcFXYmV4Xl4WRoRXAitEK5KjSzojkRdiFXhf25KiwPV4UGA9+KL9fvzyWFpAnp+R6SXpS0KD0uH2hdToRD1AnDo1CuZOiq0MzyBnkrPiRtA3wYuLOi6ZGIOCQ9zhxofU6Em6BTkqH15aqwPFwVdr3B3IoP4LNk96L9w3BW5kTYxVwVmllJDXgrPknjgd0j4idVlt9T0v9K+m9JRw20ssFcYs2quPvnizj02EOKDsMabMP4kxix8IaiwzB86bV29Nza9UP5UTtWUv489RkRMWMwC0oaQXYDiPdXaV4F/FFEPC3pUOB6SQdGRM3AXBF2OVeF7aEbh0et4z3Ve6/Y9MgnwYFuxbcNcBBwm6RHgSOAWZImRMRLEfE0QETcDTwC7FcvECfCYfC+ws5U1n2FZl2k7q34IuLZiBgbEXtExB7APODkiFgg6dXpYBsk7QXsC9T9xe9EaK4K20Q3VoU+aKY7DfJWfLVMBO6VtAj4AXBmRDxTbwHvIxwm7yvsTN5XaFasgW7FVzH/6Nz0D4EfDmVdrggNcFXYLlwVmjWeE2EDeF+hmVn7ciK0jVwV9lXWg2ZcFZo1VqkSoaQvSnpQ0r2SrpO0fY1+j0q6L11Hrt/9EovgqtDMrD2VKhECNwMHRcTrgV8Bn6rT95h0HbkJrQltYJ2QDF0V9uWqsDxcFVqzlCoRRsRN6bBZyM4L2a3IeMzMrPOVKhFW+ADw0xptAdwk6W5J0+q9iKRpkhZIWrDhpTUND7KSq8LGclVYm6tCs8Zo+XmEkm4BXlul6byI+K/U5zxgHfCdGi/z5ohYKek1wM2SHoyIudU6psv2zAAYvUNPDPsNmJlZR2l5RRgRx0XEQVUevUnw/cCfAX8ZEVUTV0SsTP8+CVxHdsuO0nBV2FiuCmvrxqrQrNFKNTQqaRLwCbJrxr1Qo89W6WaMSNoKOB7wJerNuoSHR63RSpUIga+RXVX85nRqxOUAknaR1HupnZ2A2yXdA9wF/CQiflZMuLV1QlVYJq4Ka3NVaDY8pbrWaETsU2P+48DkNL0MOLiVcXWry+Yu46yJexUdhlk/vlehNVLZKsKO4qqwsVwV1uaq0GzTORE2WbsnwzIdNFMWTobl4H2F1ihOhNZWylAVmllncSJsAVeFncdVYTm4KrRGcCK0tuOq0MwayYmwRVwVdh5XheXgqtCGy4nQ2pKrQjNrFCfCFnJV2HlcFZq1PyfCFmv3ZFgmrgqtl4dHbTicCG1IXBX256rQrL05ERbAVWHjuCq0Xq4KbVM5EdqQuSrsz1WhWWNJmiTpIUlLJZ1bpf1MSfelGzTcLmlcru1TabmHJJ0w0LqcCAviqrBxXBWadRZJI4FLgROBccBp+USXfDciXhcRhwBfAL6clh0HTAUOBCYBl6XXq8mJ0DaJq8L+XBUWz8OjHeMwYGlELIuItcA1wJR8h4jI/wLeCui9kfsU4JqIeCkifg0sZYCbt5fqNkzd5u6fL+LQYw8pOoyOMGf5cxzTs23RYZh1tf97fu1QfiSPlbQg93xGRMxI07sCj+XaVgCHV76ApLOBjwKjgbfklp1Xseyu9QJxRWibzFVhf64Ki+eqsG08FRETco8ZAy/SV0RcGhF7A58Ezt/UQJwIrWN4X6FZx1gJ7J57vluaV8s1wNs3cVknwqL5oJnO46qweK4K2958YF9Je0oaTXbwy6x8B0n75p6+DXg4Tc8CpkraXNKewL7AXfVWVrpEKOlCSSvTIbGLJE2u0a/uobXWGmUbHnVVWF83JUNrXxGxDjgHuBF4AJgZEUskTZd0cup2jqQlkhaR7Sc8PS27BJgJ3A/8DDg7ItbXW19ZD5a5JCL+pVZj7tDat5LtCJ0vaVZE3N+qABvJB810ng3jT2LEwhuKDqOrTTz1IOZeu7joMGwTRcRsYHbFvAty0x+us+xFwEWDXVfpKsJBGvDQWmsdV4XtxVWhWV9lTYTnSLpX0pWSxlRpr3ZobdXDYyVNk7RA0oINL61pRqwN4X2Fnaes+wq7ifcV2mAUkggl3SJpcZXHFODrwN7AIcAq4EvDWVdEzOg9PHfE5ls3IHqrxlVhe3FVaPaKQvYRRsRxg+kn6Qrgx1Wahnx4rFkRvK+weN5XaAMp3dCopJ1zT98BVPsED3hobTtq9yBbA1QAAAzZSURBVOFRV4XtpZuqQg+RWj2lS4TAF9IVxe8FjgH+DkDSLpJmQ+1Da4sK2Kwe7ys0K7fSJcKIeG+6ovjrI+LkiFiV5j8eEZNz/WZHxH4RsXc6VLYjuCpsLFeF9bkqNCthIjTrRK4KzcrLibCEXBU2lqvC+lwVWrdzIjRrEVeFZuXkRGhN4aqwvbgqtG7mRFhS7T48atW5KiwHJ0PLcyK0pnFV2F66qSo0y3MiLDFXhZ3JVWE5uCq0Xk6E1lSuCtuLq0LrRk6EJeeqsDO5KiwHV4UGToTWAq4KzazMnAjbgKvCzlTWqrDbhkddFZoTobWEq0IzKysnwjbRCVVh2ZJhGbgqLAdXhd3NidC6lqtCMwMnwrbiqrAzuSosB1eF3cuJ0Lqaq0LLczIsD0mTJD0kaamkc6u0T5S0UNI6Se+qaFsvaVF6zBpoXU6EbcZVYWdyVWj2CkkjgUuBE4FxwGmSxlV0+w3wfuC7VV7ixYg4JD1OHmh9ToTW9VwVWp6rwlI4DFgaEcsiYi1wDTAl3yEiHo2Ie4ENw13ZqOG+QCNJuhbYPz3dHvhdRBxSpd+jwPPAemBdRExoWZAlcPfPF3Hosf02S1u5bO4yzpq4V9FhlMqG8ScxYuENRYfRT897TmH51TOLDsPawAvPvzCUUauxkhbkns+IiBlpelfgsVzbCuDwIYSyRXrtdcDFEXF9vc6lSoQRcWrvtKQvAc/W6X5MRDzV/KisG8xZ/hzH9GxbdBhWEhNPPYi51y4uOoxO91QTi5ieiFgpaS/gVkn3RcQjtTqXcmhUkoBTgO8VHYs1j/cV9ud9hWYArAR2zz3fLc0blIhYmf5dBtwGvKFe/1ImQuAo4ImIeLhGewA3Sbpb0rQWxlUanXDQTNl4X6HleV9hoeYD+0raU9JoYCow4NGfAJLGSNo8TY8FjgTur7dMyxOhpFskLa7yyO8IPY361eCbI2I82RFFZ0uaWGd90yQtkLRgw0trGvQurFFcFfbnqtC6XUSsA84BbgQeAGZGxBJJ0yWdDCDpjZJWAO8G/l3SkrT4AcACSfcAc8j2EdZNhC3fRxgRx9VrlzQKeCdwaJ3X6C17n5R0HdkRRnNr9J0BzAAYvUNPbGLYpdQJB82UjfcVWp73FRYnImYDsyvmXZCbnk82ZFq53B3A64ayrjIOjR4HPBgRK6o1StpK0ja908DxgD+pbcxVoZWZh0g7XxkT4VQqhkUl7SKp95fBTsDtqey9C/hJRPysxTGWhvcVNl4Z9hV6eNSsdUp1+gRARLy/yrzHgclpehlwcIvDsibzeYVmVpQyVoQ2RK4KG89VYW3dWBV6eLSzORFaaXhfoZkVwYmwQ3RKVVimZOiqsDZXhdZJnAg7SKckQzOzVnIitNJxVdiXq8LycFXYmZwIO4yrQjOzoXEitFJyVdiXq8LycFXYeZwIO5CrQjOzwXMitNJyVdhXWavCbuSqsLM4EXYoV4XWKt04PGqdxYnQSs1VYV+uCsvDVWHncCLsYK4KrVVcFVo7cyK00nNV2JerQrPGciLscK4KrVW6sSr08GhncCLsAk6GjeWqsLZuTIbW/pwIrS2UaXjULM9VYftzIuwSrgoby1Vhba4Krd04EVrbcFVoZeWqsL05EXYRV4WN5aqwNleF1k4KSYSS3i1piaQNkiZUtH1K0lJJD0k6ocbye0q6M/W7VtLo1kRuRXNVaGXlqrCxJE1KeWCppHOrtG+evv+XpnywR65twDySV1RFuBh4JzA3P1PSOGAqcCAwCbhM0sgqy38euCQi9gFWA2c0N9zO4aqwsVwV1uaq0DZV+t6/FDgRGAeclvJD3hnA6pQHLiHLC0PJIxsVkggj4oGIeKhK0xTgmoh4KSJ+DSwFDst3kCTgLcAP0qxvAm9vZrxWLq4KraxcFTbMYcDSiFgWEWuBa8jyQ94Usu9/yPLBsSk/DJhHKo1qaOjDtyswL/d8RZqXtyPwu4hYV6fPRpKmAdPS05dWzTxrcYNibYaxwFPNXsmPZw5r8ZbEOJAf124qRXwDcIzDV/b4oPwx7t/oF3x59W9uXDXzrLGD7L6FpAW55zMiYkaa3hV4LNe2Aji8YvmNfSJinaRnyfLDYPJIH01LhJJuAV5bpem8iPivZq23UtqwM1JMCyJiwgCLFKbs8UH5Yyx7fOAYG6Hs8UH5Y6xIQg0REZMa/Zqt0LREGBHHbcJiK4Hdc893S/Pynga2lzQqVYXV+piZWfsaTC7o7bNC0ihgO7L8MJhl+yjb6ROzgKnpaKA9gX2Bu/IdIiKAOcC70qzTgZZVmGZm1nTzgX3TGQKjyQ5+mVXRZxbZ9z9k+eDWlB8GzCOVijp94h2SVgBvAn4i6UaAiFgCzATuB34GnB0R69MysyXtkl7ik8BHJS0lGxP+xiBXPWPgLoUqe3xQ/hjLHh84xkYoe3xQ/hhLG18a7TsHuBF4AJgZEUskTZd0cur2DWDHlAc+Cpyblq2ZR2pRlkDNzMy6U9mGRs3MzFrKidDMzLpaxyXCdrp8W3r9RenxqKSql31Jbfelfg0/5HmAGC+UtDIX5+Qa/epeDqmJ8X1R0oOS7pV0naTta/Rr+TYcziWiWhDb7pLmSLo//X/5cJU+R0t6Nve3v6BV8eViqPt3U+araRveK2l8i+PbP7d9Fkl6TtJHKvq0dDtKulLSk5IW5+btIOlmSQ+nf8fUWPb01OdhSadX69ORIqKjHsABZCeK3gZMyM0fB9wDbA7sCTwCjKyy/Exgapq+HPibFsX9JeCCGm2PAmML2p4XAh8foM/ItD33Akan7TyuRfEdD4xK058HPl+GbTiYbQKcBVyepqcC17Ywvp2B8Wl6G+BXVeI7GvhxEZ+7wf7dgMnATwEBRwB3FhjrSOC3QE+R2xGYCIwHFufmfQE4N02fW+3/CbADsCz9OyZNjyny79+qR8dVhNGGl29L6z0F+F6z19Ukg7kcUlNExE3xylWG5pGdM1QGw7lEVNNFxKqIWJimnyc7Mq/u1TdKagrwrcjMIzvHeOeCYjkWeCQilhe0fgAiYi7wTMXs/Get1vfaCcDNEfFMRKwGbia7VmfH67hEWEe1S/YM6/JtDXQU8EREPFyjPYCbJN2t7JJxrXZOGna6ssaQymC2bSt8gKw6qKbV23Aw26TPJaKA3ktEtVQakn0DcGeV5jdJukfSTyUd2NLAMgP93cry2YOsqq/1Y7bo7bhTRKxK078FdqrSp0zbsqXKdq3RQVFJLt82GIOM9TTqV4NvjoiVkl4D3CzpwfSrr+kxAl8HPkv2hfRZsiHcDzRq3YMxmG0o6TxgHfCdGi/T1G3YriRtDfwQ+EhEVN5KYyHZMN+atG/4erKTk1upLf5u6ViCk4FPVWkuw3bcKCJCks+by2nLRBhtdPm2gWJVdmmgdwKH1nmNlenfJyVdRzbs1rAvg8FuT0lXUP1610O+pNFQDGIbvh/4M+DYSDs7qrxGU7dhFcO5RFRLSNqMLAl+JyJ+VNmeT4wRMVvSZZLGRkTLLiQ9iL9bUz97Q3AisDAinqhsKMN2BJ6QtHNErEpDx09W6bOSbH9mr93IjrXoeN00NFrWy7cdBzwYESuqNUraStI2vdNkB4e07A4aFftb3lFj3YO5HFKz4psEfAI4OSJeqNGniG04nEtENV3aF/kN4IGI+HKNPq/t3Wcp6TCy74tWJurB/N1mAe9LR48eATybGwJspZqjOkVvxyT/Wav1vXYjcLykMWkXyPFpXucr+midRj/IvqxXAC8BTwA35trOIzuS7yHgxNz82cAuaXovsgS5FPg+sHmT470KOLNi3i7A7Fw896THErLhwFZuz28D9wH3kv1n2rkyxvR8MtmRh4+0Msb0d3oMWJQel1fGV9Q2rLZNgOlkSRtgi/QZW5o+c3u1cLu9mWy4+97ctpsMnNn7eSS7xNWStN3mAX/S4s9e1b9bRYwiu4HrI+lzOqGVMaYYtiJLbNvl5hW2HckS8irg5fRdeAbZvuefAw8DtwA7pL4TgP/ILfuB9HlcCvxVq7dlUQ9fYs3MzLpaNw2NmpmZ9eNEaGZmXc2J0MzMupoToZmZdTUnQjMz62pOhGYtJulnkn4nqdrFCcysxZwIzVrvi8B7iw7CzDJOhGZNIumN6WLlW6SrpCyRdFBE/Bx4vuj4zCzTltcaNWsHETFf0izgc8CWwNUR0bLL45nZ4DgRmjXXdLLrjv4B+FDBsZhZFR4aNWuuHYGtye4Cv0XBsZhZFU6EZs3178Cnye6T+PmCYzGzKjw0atYkkt4HvBwR35U0ErhD0luAfwT+GNha0grgjIjojtvdmJWQ7z5hZmZdzUOjZmbW1ZwIzcysqzkRmplZV3MiNDOzruZEaGZmXc2J0MzMupoToZmZdbX/Dzl1GtXSXcp4AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 1152x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plotModel(predict_no_hidden, t='fcnn separation without hidden layer')\n",
    "plt.scatter(x1,x2,c=\"black\",s=50)\n",
    "predict_no_hidden((x1,x2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Forwad pass with hidden layer (matrix multiplication)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "# we use the same values for x1 and x2 and random normal values for the weights\n",
    "X=np.array([[x1,x2]])\n",
    "np.random.seed(22)\n",
    "W1=np.reshape((np.random.normal(0,1,16)),(2,8))\n",
    "np.random.seed(22)\n",
    "b1=np.reshape((np.random.normal(0,1,8)),(8,))\n",
    "np.random.seed(22)\n",
    "W2=np.reshape((np.random.normal(0,1,8)),(8,1))\n",
    "np.random.seed(22)\n",
    "b2=np.reshape((np.random.normal(0,1,1)),(1,))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(1, 2)\n",
      "(2, 8)\n",
      "(8,)\n",
      "(8, 1)\n",
      "(1,)\n"
     ]
    }
   ],
   "source": [
    "print(X.shape)\n",
    "print(W1.shape)\n",
    "print(b1.shape)\n",
    "print(W2.shape)\n",
    "print(b2.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.76752454, 0.01533699, 0.90265373, 0.27163597, 0.57707969,\n",
       "        0.41351693, 0.37964327, 0.52893641]])"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hidden=sigmoid(np.matmul(X,W1)+b1)\n",
    "hidden"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.44860787]])"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "p_out=sigmoid(np.matmul(hidden,W2)+b2)\n",
    "p_out"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "## funtion to return the probaiblity output after the hidden layer \n",
    "def predict_hidden(X):\n",
    "    hidden=sigmoid(np.matmul(X,W1)+b1)\n",
    "    return(sigmoid(np.matmul(hidden,W2)+b2))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(array([[1. , 2.2]]), array([[0.44860787]]))"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X,predict_hidden(X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(100, 1, 1)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.PathCollection at 0x7f34b73c9630>"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAcIAAAEWCAYAAAD1t5d8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3de7xVdZ3/8ddbFAWkVEhTNISkC9aoSViTFZYpMiV2M6wcbw3ZxC9nmqbsZkY1YzVdnJ82RJOWWRlNXkhJpTF1zDTQ8YaXRMLgiBheUdBEPvPH+h5dbPc+Z59z9l577b3fz8djP9h7re9a63vW2ez3+XzXZSsiMDMz61ZbtboDZmZmreQgNDOzruYgNDOzruYgNDOzruYgNDOzruYgNDOzruYg7HCSXi7pJknrJX2s1f0pO0nLJE0r6/YlXSnpQ3Wua5qk1X3Mnyfp833MD0l71Zh3rKRr6unHQEjaM21360av26wWv9k63yeB30TEvq3uSNlI+gGwOiI+1zstIvZuXY+23L6kU4G9IuKDTdrWic1Yr1m7cUXY+cYDy1rdiaK5orB6KePPwi7mX34Hk3QFcBBwhqTHJb1M0ghJ35B0r6RHJV2TpvUOSR0j6U+S1kn6bG5dp0paIOmcNMy6TNKUGtuVpG9JekDSY5JulfSqNG9bSf+WtrE2Dc+NSPOmSVot6TNp+yslfSC33r+R9L9pnatSxdQ7r7f/J0j6E3BFmv5zSfenn/VqSXun6bOBDwCfTPvml2n6SkkH5/r6bUn3pce3JW1b0dd/Sj/nGknH1dgfB0m6Nfd6saQludf/I+mI/PYlTQc+A7wv9e/m3CrHS/pt+j1cLmlsP++Dqn2U9ANJX869/ufU5j5Jx1esY4ykhWnf/x54acX8V6Sf6yFJd0k6smI7Z0q6JPX5eklbLN9H34+TdEdaboWkD+fm3SbpHbnX26T3zX7p9eskXSvpEUk3KzfkrGyI+SuSfgtsACbW0x/rUBHhRwc/gCuBD+Ven5mmjQOGAX8NbAvsCQTwPWAEsA/wFPDKtNypwJPAjLTcvwLX1djmocANwA6AgFcCu6Z53wIWAjsBo4FfAv+a5k0DNgHfTH16M/AE8PLc/FeT/QH3V8Ba4Ig0r7f/5wCjgBFp+vFpO9sC3wZuyvXzB8CXK/q+Ejg4PZ8LXAfsDLwIuBb4UkVf5wLbpP2yAdixyv4Ykfbd2NR2LdCT+jUC2AiMqbL9U4Fzq/w+7wFelpa9Ejitxu+hzz7mf35geurXq9L++0nan3ul+ecBC9K8V6X+X5PmjQJWAceRHW7ZD1gHTM5t50Fgapr/Y+C8Gn3u/T1unV7/DVnoiuz9sAF4TZr3SeBnuWVnArem5+PSNmeQvV/ell6/KLcf/wTsnfq0Tav/r/rRuocrwi6ibPjneOCkiOiJiGci4tqIeCrX7IsRsTEibgZuJgvEXtdExKKIeAb4UcW8vKfJPuRfASgi7oiINZIEzAb+MSIeioj1wL8AsyqW/3xEPBURVwGXAEcCRMSVEXFrRGyOiFuAn5J9OOadGhFPRMTGtMxZEbE+/YynAvtIemGdu+wDwNyIeCAi/gx8ETi64uecGxFPR8Qi4HHg5ZUrSX1ZArwJ2J9sv/4WeAPwOuDuiHiwzj4BnB0Rf0jrXQD0dfy3rj6S7eOzI+K2iHiCbF8BIGkY8G7glLRvbwN+mFv27cDKiDg7IjZFxP8CvwDem2tzQUT8PiI2kQVhXcesI+KSiLgnMlcBlwNvTLPPBWZIekF6fTTZ+xLgg8Ci9H7dHBGLgaVkwdjrBxGxLPX56Xr6Y53Jx1G6y1hgO7KKopb7c883ANv3MW87SVunD7dnRcQVks4gqz7HSzof+ETa9kjghiwTgewv/WG5xR9OH8S97gV2A5B0AHAaWUUynKzK+3lF/1c9u+LsA/wrZB/ILwI2p1ljgUdr/Px5u6XtP68vyYMVP3vl/sq7iqxCW52eP0wW4k+l1wPR1++oUr193I2siu+V/7lfRPZZsarG/PHAAZIeyU3bmudCaaB9fpakw4AvkFXAW5G9f24FiIj70tDmuyVdABwGnJTr03vzQ6dkVfFvcq/zP491MVeE3WUd2RBdXcdnhiIi/j0i9gcmk32I/XPa/kZg74jYIT1eGBH5D8UdJY3KvX4JcF96/hOyYdU9IuKFwDyyIN1i07nn7ycbLjsYeCHZsBu5Zfr76pX7yD5Qq/VloHqD8E3p+VVkQfhmagdhkV8NswbYI/f6JbnnfyYbYq01fxVwVe53ukNEbB8RHxlKh9Lx2F8A/wbsEhE7AIvY8nf+Q7Lq773A7yKiJ9enH1X0aVREnJZb1l+9Y4CDsKtExGbgLOCbknaTNEzS63tPAGkUSa+VdICkbciO8T0JbE7b/x7wLUk7p7bjJB1asYovShou6Y1kw269Vd9o4KGIeFLSVLKg68tosorrQbJK4l8q5q+l75Mkfgp8TtKL0gkpp5ANxw3GtWRDklOB30fEMlIlBVxdY5m1wJ4q5ozGBcCxkiZLGklWhQGQhsLPB06VNFLSZOCY3LIXAy+TdHQ6YWWb9B545RD71Fv1/xnYlKrDQyraXAi8hqwSPCc3/VzgHZIOTe/z7dIJTrsPsU/WgRyE3ecTZENLS4CHgK/S+PfBC8gC72GyIbQHga+neZ8ClgPXSXoM+DVbHrO6Py13H9mxpBMj4s407++BuZLWk4XSgn76cU7afg9wO9mJL3nfByanswovrLL8l8mOK91Cts9uTNMGLA333ggsi4i/pMm/A+6NiAdqLNb7B8CDkm4czHYH0L9fkZ1MdAXZ7+eKiiZzyIYz7yc7+eXs3LLryQJqFtnv7X6y99WQ/sBK6/0Y2e/5YbI/fBZWtNlIVjVOIAvr3umryEYDPkMWpKvIRiX8mWfPowiPDlg5pNPbz40I/9VudZN0CvCyaNKNB6zz+WQZM2tbknYCTmDLs3nNBqSlwwSSzkoX+t6Wm7ZTujD37vTvjjWWPSa1uVvSMdXamFnnkvR3ZEOev4qIWsdZrU1Jmp5uzrBc0slV5o+X9N+Sbkk3SNg9N29A+dDSoVFJbyK7rumciOi988jXyE6IOC398DtGxKcqltuJ7NjNFLIzv24A9o+Ihwv9AczMrOHSpU9/ILsRwmqycxqOiojbc21+DlwcET+U9BbguIg4ejD50NKKMP0V91DF5Jk8d7HuD4Ejqix6KLA4XZT9MLCY7M4YZmbW/qYCyyNiRTq57DyybMibzHMndf0mN3/A+VDGY4S7RMSa9Px+YJcqbcax5cWwq9O051F2T8nZANp62/23fsGuDeyqWXvZYYeRre5CW9l5++FN38Z2W9zYaWiefnBd3W1vue/P6yLiRQ3bOLDP8O1jfTxTV9s/bnpyGdmlVb3mR8T89LzaZ/wBFau4GXgXcDrwTmC0pDE1lq2aD73KGITPioiQNKSx27Rj5wMMHzMhdpnxhX6WMOtMM494Tau70FbmHLhn07cxaeOKhq1r7bnfHVD7cZ+fd2//rQZmfTzDl3eYUFfbD6y748mIqHrj/jp9guwLBY4luxa3B6gvhSuUMQjXSto13ZtyV6DaNVY9ZHfp6LU72U10zcyGrNkh2MoAbBM9bHkno93TtGdFxH1kFSGStgfeHRGPSBpwPpTx4tKFPHfXimOAi6q0uQw4RNKO6azSQ9I0M6vC1WB95hy4p0OwHJYAkyRNkDSc7GYNW9xMQdLY3F2XPk121ywYRD60tCKU9FOy5B4raTXZbZ1OAxZIOoHsriBHprZTyO4y8qGIeEjSl8h2FmR316886cbMrG4OwPKIiE2S5pAF2DDgrIhYJmkusDQiFpJlx7+mw2dXAx9Nyw44H7rqzjI+RmjdyNVg/7oxBMd9ft4NQzxG9zwTtxkRAzhG2PDtD1YZjxGamRWiGwPQnq+MxwjNzJrOIWi9XBGadTAPi1bXLiHoACyGg9DMuoYD0Krx0KhZh3I1uCWHoNXiitDMOl4zQ9AB2P5cEZp1IFeDz3EIWn9cEZpZR2qHodBWBOCVp19V+DbLzkFo1mFcDboKrMUhWJ2D0Mw6StlD0AFYPg5CM+sIZQ9AKD4EHYD1cRCadZBuHRYtewi6Ciw3B6GZtS2fEPN8DsCBcxCadYhuqwZdBW7JATh4vo7QzNqOQ3BLDsGhcUVo1gG6pRr0UOiWHICN4SA0s7bgKnBLDsHGcRCaWek5BJ/jAGy8UgahpJcDP8tNmgicEhHfzrWZBlwE/DFNOj8i5hbWSbOS6ORhUQfgcxyAzVPKIIyIu4B9ASQNA3qAC6o0/Z+IeHuRfTMrE4fg4DgELa+UQVjhrcA9EXFvqztiZsVoVgg6AK2adrh8Yhbw0xrzXi/pZkm/krR3kZ0ya7VOrQYdghmHYHFKXRFKGg4cDny6yuwbgfER8bikGcCFwKQq65gNzAYYNmpME3trVpxODEEHYMYBWLxSByFwGHBjRKytnBERj+WeL5L0HUljI2JdRbv5wHyA4WMmRLM7bGYD5xB0ALZS2YPwKGoMi0p6MbA2IkLSVLJh3geL7JyZDV0zQrCdAhAcgq1W2iCUNAp4G/Dh3LQTASJiHvAe4COSNgEbgVkR4YrPOl4nDYt2ewg6AMuhtEEYEU8AYyqmzcs9PwM4o+h+mdnQdXsAgkOwTEobhGb2fJ1QDXZ7CDoA6yNpOnA6MAz4z4g4rWL+t4CD0suRwM4RsUOa9wxwa5r3p4g4vK9tOQjNrDBlDEFXgeWTbqRyJtnhsdXAEkkLI+L23jYR8Y+59v8P2C+3io0RsW+923MQmlkhGh2CrgI72lRgeUSsAJB0HjATuL1G+6OALwx2Yw5CszbRrsOirgIdgoMwDliVe70aOKBaQ0njgQnAFbnJ20laCmwCTouIC/vamIPQzJqmbCHoAGyu0TuPZtpH3lxf48/fMTaFVa/56brvgZoF/FdEPJObNj4ieiRNBK6QdGtE3FNrBQ5CM2uKsg2Fehi0dNZFxJQa83qAPXKvd0/TqpkFfDQ/ISJ60r8rJF1JdvzQQWjWztptWLRMIegqsC0tASZJmkAWgLOA91c2kvQKYEfgd7lpOwIbIuIpSWOBNwBf62tjDkIza5gyBSC4CmxXEbFJ0hzgMrLLJ86KiGWS5gJLI2JhajoLOK/iZiqvBL4raTPZHcdOy59tWo2D0Kzk2qUaLFMIugpsfxGxCFhUMe2UitenVlnuWuDVA9mWg9DMhqwbQ9AB2DkchGY2JI0MwXYYCnUAdh4HoVmJlXlY1FWgdQoHoZkNWLeFoAOwszkIzUqqrNWgh0Kt0zgIzawurgKtUzkIzaxf3VQFOgC7j4PQrITKNCxalhB0FWjN4iA0K5lODEFXgVZmDkKzEnEIbslVoBWhtEEoaSWwHngG2FR5l3JJAk4HZgAbgGMj4sai+2nWiRoRgq4CrV2UNgiTgyJiXY15hwGT0uMA4D+o8cWNZu2gDNWgq0DrRmUPwr7MBM5Jdx2/TtIOknaNiDWt7pjZQDkEM64CrRXKHIQBXC4pgO9W+ebiccCq3OvVadoWQShpNjAbYNioMc3rrVkba/VQqKtAa6UyB+GBEdEjaWdgsaQ7I+Lqga4kBeh8gOFjJkQ/zc0K1+pqsNND0AFo/SltEEZET/r3AUkXAFOBfBD2AHvkXu+eppm1jXYPwTIHIDgErT6lDEJJo4CtImJ9en4IMLei2UJgjqTzyE6SedTHB83q18kh6AC0gShlEAK7ABdkV0iwNfCTiLhU0okAETGP7JuLZwDLyS6fOK5FfTUblFZWg60KQVeBVkalDMKIWAHsU2X6vNzzAD5aZL/MGqVdQ9BVoHWiUgahWSfrthB0FWhl5yA06wKdOhTqALRGcBCaFagV1aCrQLO+OQjNCuIQbAwHoDWag9CsQxUdgq4CrV05CM0KUHQ1ONgQdBVo3WirVnfArNM5BIfGIWjN5orQrIk6OQQdgNYpXBGadQiHoNnguCI0a5KiqsFOGgp1AFovSdOB04FhwH9GxGlV2hwJnEr2tX03R8T70/RjgM+lZl+OiB/2tS0HoVkTdGIIugq0okgaBpwJvI3su2aXSFoYEbfn2kwCPg28ISIeTl/Zh6SdgC8AU8gC8oa07MO1tucgNGtTnRKCDkCrYiqwPN13mvQtQzOB23Nt/g44szfgIuKBNP1QYHFEPJSWXQxMB35aa2MOQrMGK6IaHEwIeijUmm2bMWPZ5YMfrq/x5+eNlbQ0N2V++iJ1gHHAqty81WRft5f3MgBJvyUbPj01Ii6tsey4vrriIDRrM0WFoAPQmmxdREwZwvJbA5OAaWRfzH61pFcPdkVm1iYGGoKuAq1N9QB75F7vnqblrQauj4ingT9K+gNZMPaQhWN+2Sv72piD0KyBmjks2u4h6AC0AVgCTJI0gSzYZgHvr2hzIXAUcLaksWRDpSuAe4B/kbRjancI2Uk1NTkIzdpAESHoKtDKIiI2SZoDXEZ2/O+siFgmaS6wNCIWpnmHSLodeAb454h4EEDSl8jCFGBu74kztTgIzRqkWdXgQELQVaB1iohYBCyqmHZK7nkAH0+PymXPAs6qd1ulC0JJewDnALuQXQMyPyJOr2gzDbgI+GOadH5EzC2yn2ZFaHYIugo0K2EQApuAf4qIGyWNJrsYcnH+QsrkfyLi7S3on1kh2jUEHYDWbkoXhBGxBliTnq+XdAfZNSCVQWhWGo0eFq0Wgk88vp5FF53PvStWMH7iRGbMfBf7DvvzoNbvEDR7TumCME/SnsB+wPVVZr9e0s3AfcAnImJZjXXMBmYDDBs1pjkdNWugaiF4w/W/48MfeA+bN29m44YNjBg5kq+fcjK/PPsMDnztfnWv2wFo9nyl/fYJSdsDvwD+ISIeq5h9IzA+IvYB/j/ZabRVRcT8iJgSEVO22nZ08zps1gC1KsEPf+A9PPH442zcsAGAjRs2sP6JDbzjuDk8/sSGutbtEDSrrpRBKGkbshD8cUScXzk/Ih6LiMfT80XANuk6ErPCNWpYtNYxwUUXnc/mzZurztscwYKLL+t33Q5Bs9pKNzQqScD3gTsi4ps12rwYWBsRIWkqWaA/WGA3zRqqrxNj7l2x4tlKsNITGzay/N5VVeeBzwo1q0fpghB4A3A0cKukm9K0zwAvAYiIecB7gI9I2gRsBGala0rMCtWIarC/s0PHT5zIiJEjq4bhqJEj2Gv8HlWWchVoVq/SBWFEXAOonzZnAGcU0yOz5qnnEokZM9/F1085ueq8rSSOfPuhW0xzFWg2MKU8RmjWDeq9TnDfYX/ml2efwehRIxk1cgSQVYKjR43kl2efwfajRj7b1iFoNnClqwjN2sVQhkXrDcHeC+UPfO1+rLp+MQsuvozl965ir/F7cOTbD3UImjWAg9CsYAMNwV7bjxrJ8e975/PaNTMAwSFonc9BaFagwYZgLa4CzYbOxwjNBmEww6IOQbNychCaFcAhaFZeDkKzARpoNegQNCs3B6FZEzkEzcrPQWjWJA5Bs/bgs0bNBqDeYdF2CUEHoJkrQrOu5RA0yzgIzerUSdWgQ9DsOQ5Csy7jEDTbkoPQrA6dUg06BM2ezyfLmPWjUd9A30oOQLPaXBGa9WEgIdjoarBRHIJmfXMQmtXQ6kqwEcOiDkGz/jkIzapo1m3UiuQQNKtPn0Eo6QWSXlpl+l81r0vPbmO6pLskLZd0cpX520r6WZp/vaQ9m90n6w7NrASLOknGIWjtrr8MyLV7t6SQNCW93lPSRkk3pce8/rZV82QZSUcC3wYekLQNcGxELEmzfwA07dNC0jDgTOBtwGpgiaSFEXF7rtkJwMMRsZekWcBXgfc1q0/WHZr59UpFcABaJ6gzA5A0GjgJuL5iFfdExL71bq+vivAzwP5pZccBP5LU+/XYqncDgzQVWB4RKyLiL8B5wMyKNjOBH6bn/wW8VVKz+2UdrNnHBJt9koxD0DpIPRkA8CWyIujJoWysryAcFhFrACLi98BBwOckfQyIoWy0DuOAVbnXq9O0qm0iYhPwKDCmckWSZktaKmnp5qfWN6m71u4GG4LNqAYHMyzqELQ2NLb3szk9Zufm9ZsBkl4D7BERl1RZ9wRJ/yvpKklv7K8jfV1HuF7SSyPiHoCIWCNpGnAhsHd/Ky6LiJgPzAcYPmZCswPc2lARZ4c2sxp0CFpZPKltuXvExHqbr4uIKYPZjqStgG8Cx1aZvQZ4SUQ8KGl/4EJJe0fEY7XW11dF+BFgK0mTeydExHpgOvChwXR+AHqAPXKvd0/TqraRtDXwQuDBJvfLOsxQQrAM1aBD0DpUfxkwGngVcKWklcDrgIWSpkTEUxHxIEBE3ADcA7ysr43VDMKIuDki7gYWSPqUMiPIUvjvB/5zDcgSYJKkCZKGA7OAhRVtFgLHpOfvAa6ICFd8VreirhNsVjXoELQO1mcGRMSjETE2IvaMiD2B64DDI2KppBelk22QNBGYBPT5n7Ce6wgPIEvma1Pn7gPeMPCfq37pmN8c4DLgDmBBRCyTNFfS4anZ94ExkpYDHwdqnl5rVmmoIdjqatAhaJ2szgyo5U3ALZJuIjuR8sSIeKivBeq51+jTwEZgBLAd8MeI2FzHckMSEYuARRXTTsk9fxJ4b7P7YZ2nyDvGFH07NbNO0V8GVEyflnv+C+AXA9lWPRXhErIgfC3wRuAoST8fyEbMyqIRIdjq6wZdDZo1Vj0V4QkRsTQ9XwPMlHR0E/tk1hStvndoX5rx5btmVp9+K8JcCOan/ag53TFrjkaF4ECqQQ+LmrUH33TbOl6ZK0HwSTJmreYgtI7WyBB0NWjWmRyE1rHKXgkOlKtBs+ZwEFpHamUIDqQa9EkyZq3nIDSrQ6svmTCz5nEQWsdpdDXoEDTrbA5C6yiddlzQzJrPQWjWh4FWgz5b1Kz9OAitY7gaNLPBcBCa1dDsY4M+Y9SsHByE1hHKcIKMh0XN2pOD0Nqeh0TNbCgchGYVfLmEWXdxEFpbK8OQKHhY1KydOQjNzKyr1fPFvIWR9HXgHcBfgHuA4yLikSrtVgLrgWeATRExpch+WjmUpRo0s/ZWtopwMfCqiPgr4A/Ap/toe1BE7OsQtHbkSyfMyqNUQRgRl0fEpvTyOmD3VvbHyqtM1aCPD5q1t1IFYYXjgV/VmBfA5ZJukDS7r5VImi1pqaSlm59a3/BOWvHKFIJm1v4KP0Yo6dfAi6vM+mxEXJTafBbYBPy4xmoOjIgeSTsDiyXdGRFXV2sYEfOB+QDDx0yIIf8AZmbWUQoPwog4uK/5ko4F3g68NSKqBldE9KR/H5B0ATAVqBqE1lnKVg16WNSs/ZVqaFTSdOCTwOERsaFGm1GSRvc+Bw4Bbiuul2Zm1klKFYTAGcBosuHOmyTNA5C0m6RFqc0uwDWSbgZ+D1wSEZe2prtWpLJVg2bWGUp1HWFE7FVj+n3AjPR8BbBPkf2y1uukEPSlE2blUraK0Kxt+PigWfNImi7pLknLJZ1cZf6Jkm5No4fXSJqcm/fptNxdkg7tb1sOQiu9TqoGzax/koYBZwKHAZOBo/JBl/wkIl4dEfsCXwO+mZadDMwC9gamA99J66vJQWhdxSFo1hamAssjYkVE/AU4D5iZbxARj+VejiK7vpzU7ryIeCoi/ggsT+urqVTHCM0q+bsGzTrWWElLc6/np+u+AcYBq3LzVgMHVK5A0keBjwPDgbfklr2uYtlxfXXEQWil5SHRLU076c1cefpVre6GWU0PPP4XzrhmZb3N1w31XtERcSZwpqT3A58DjhnMejw0amZmZdMD7JF7vXuaVst5wBGDXNZBaOUz84jXdHQ1uMsHP9zqLpiV3RJgkqQJkoaTnfyyMN9A0qTcy78B7k7PFwKzJG0raQIwieya85ochFYqPibYt2knvbnVXTBruvQtRHOAy4A7gAURsUzSXEmHp2ZzJC2TdBPZccJj0rLLgAXA7cClwEcj4pm+tudjhFYazQrBMlWDZlafiFgELKqYdkru+Ul9LPsV4Cv1bssVoZWCK0EzaxUHobVUM44H5nViNejhUbPGchBay7gKNLMycBBaSxQRgmWuBn3mqFl5+GQZK5SrQDMrG1eEVpgiQ7DM1WAj+DihWeM4CK0QDsHGcxiaNYaD0JrOw6HN4zA0GzoHoTVNsy+NqKadqkGfMGNWDqULQkmnSupJ3zp8k6QZNdr1+e3F1lquAovjqtBsaEoXhMm3ImLf9FhUObPOby+2FmlVCLZTNdhoDkOzwStrEPan328vtuK1YijUnuMwNBucsgbhHEm3SDpL0o5V5lf79uKq30AsabakpZKWbn5qfTP6arR+KLSbq8E8h6HZwLUkCCX9WtJtVR4zgf8AXgrsC6wBvjGUbUXE/IiYEhFTttp2dAN6b5VaHYLtrBknzDgMzQamJXeWiYiD62kn6XvAxVVmDfgbiK05yhCCrgbNbChKNzQqadfcy3cCt1Vp1u+3F1tz+XhgubkqNKtf6YIQ+JqkWyXdAhwE/COApN0kLYLa317cqg53mzIFoKvB2hyGZvUp3U23I+LoGtPvA2bkXj/v24ut+RyC7WXaSW/mytOvanU3zEqtjBWhlVSZQtDq58rQrG8OQuuXjwc2l2+1ZtZaDkLrU1kD0MOiA+Oq0Kw2B6HVVNYQ7ESuCs1ax0FoVTkE+3f3iImt7sKAuCo0q85BaFtoh+OBnTos6qrQrDUchPassgegDZ2rQrPncxAa0D4h2KnVYC9XhWbFcxBa24SgNYarQrMtOQi7WDscD+xGrgrNQNJ0SXdJWi7p5Crz3yTpRkmbJL2nYt4zkm5Kj37vQ+0g7FLtGICdPixaJFeFVmaShgFnAocBk4GjJE2uaPYn4FjgJ1VWsTEi9k2Pw/vbnoOwC7VjCJZVu11CYdYmpgLLI2JFRPwFOA+YmW8QESsj4hZg81A35iDsIh4KbR9FDI+6KrQSGwesyr1enabVaztJSyVdJ+mI/hqX7tsnrDnaPQA9LGpWfo88soGLLryx3uZjJS3NvZ4fEfMb1JXxEdEjaSJwhaRbI+KeWo1dEXY4V4Hty1Whdbh1ETEl98iHYA+wR+717mlaXSKiJ/27ArgS2K+v9g7CDtYpAWi+JvMAAAr2SURBVOhq0KzrLAEmSZogaTgwC+j37E8ASTtK2jY9Hwu8Abi9r2UchB2qU0Kw27kqtG4UEZuAOcBlwB3AgohYJmmupMMBJL1W0mrgvcB3JS1Li78SWCrpZuA3wGkR0WcQ+hhhh3EAmlkniIhFwKKKaafkni8hGzKtXO5a4NUD2VapKkJJP8tdBLlS0k012q2UdGtqt7Ram27UiSHYDsOizb6EwlWhWXOVqiKMiPf1Ppf0DeDRPpofFBHrmt+r9tCJIWhmVoRSVYS9JAk4Evhpq/tSdj4rtDu4KjRrnlIGIfBGYG1E3F1jfgCXS7pB0uwC+1UqnR6A7TAsambtr/ChUUm/Bl5cZdZnI+Ki9Pwo+q4GD0wXS+4MLJZ0Z0RcXWN7s4HZAMNGjRlCz8uj0wPQWmfaSW/mytOvanU3zApVeBBGxMF9zZe0NfAuYP8+1tF7seQDki4guy9d1SBMF2nOBxg+ZkIMstul0S0h6Grw+Xb54IdZe+53W90Ns45TxqHRg4E7I2J1tZmSRkka3fscOAS4rcD+tUy3hKC1lo8VWrcpYxDOomJYVNJuknqvJ9kFuCZdLPl74JKIuLTgPhaq206IacdqsKhvofB3FZo1XqkunwCIiGOrTLsPmJGerwD2KbhbLdNNAWhm1gplrAgt6cYQbMdqsGi+lMKssRyEJdWNIWhm1goOwpLptuOBea4Gy8VVoXULB2GJdGsA2sD5pBmzxnEQloRDsP0VdeaomTWWg7AEHIIeFh0MnzRj1hilu3yimzgAzcxazxVhizgEn+Nq0MxayUHYAg5BaxQPj5oNnYdGC+QA7Hx3j5jIpI0rWt0NMxsAV4QFcQhW52HRofOlFGZD4yBssm6+QN46h4dHrZM5CJvIAdg3V4NmVgYOwiZxCFqRfNKM2eD5ZJkGcwDWx9WgmZWFK8IGcggatO5Waz5pxmxwHIQN4BNirFt4eNQ6kYNwiByAA+dhUTMrEwfhEDgErWw8PGo2cC0JQknvlbRM0mZJUyrmfVrSckl3STq0xvITJF2f2v1M0vBiep7xUOjguRpsfx4etSJImp5yYLmkk6vM3zZ9/i9PebBnbl6/OZLXqorwNuBdwNX5iZImA7OAvYHpwHckDauy/FeBb0XEXsDDwAnN7e5zHIBmZs2VPvfPBA4DJgNHpXzIOwF4OOXAt8hyYSA58qyWBGFE3BERd1WZNRM4LyKeiog/AsuBqfkGkgS8BfivNOmHwBHN7O+znXMIWp1a+SW9Hh61DjAVWB4RKyLiL8B5ZPmQN5Ps8x+yPHhryod+c6RS2a4jHAdcl3u9Ok3LGwM8EhGb+mjzLEmzgdnp5VOrzz3utsF27sxzB7tk3cYC65q+laEZUh/PbGBHauj4fViQsvex7P2D8vfx5Y1e4dMPrbxs9bnHja2z+XaSluZez4+I+en5OGBVbt5q4ICK5Z9tExGbJD1Klg/15MgWmhaEkn4NvLjKrM9GxEXN2m6ltGPnpz4tjYgp/SzSMmXvH5S/j2XvH7iPjVD2/kH5+1gRQg0REdMbvc4iNC0II+LgQSzWA+yRe717mpb3ILCDpK1TVVitjZmZta96sqC3zWpJWwMvJMuHepbdQtkun1gIzEpnA00AJgG/zzeIiAB+A7wnTToGKKzCNDOzplsCTEpXCAwnO/llYUWbhWSf/5DlwRUpH/rNkUqtunzinZJWA68HLpF0GUBELAMWALcDlwIfjYhn0jKLJO2WVvEp4OOSlpONCX+/zk3P779JS5W9f1D+Ppa9f+A+NkLZ+wfl72Np+5dG++YAlwF3AAsiYpmkuZIOT82+D4xJOfBx4OS0bM0cqUVZgJqZmXWnsg2NmpmZFcpBaGZmXa3jgrCdbt+W1n9TeqyUdFONdisl3ZraNfyU5376eKqknlw/Z9Ro1+ftkJrYv69LulPSLZIukLRDjXaF78Oh3CKqgL7tIek3km5P/19OqtJmmqRHc7/7U4rqX64Pff7elPn3tA9vkVToXS8kvTy3f26S9Jikf6hoU+h+lHSWpAck3ZabtpOkxZLuTv/uWGPZY1KbuyUdU61NR4qIjnoAryS7UPRKYEpu+mTgZmBbYAJwDzCsyvILgFnp+TzgIwX1+xvAKTXmrQTGtmh/ngp8op82w9L+nAgMT/t5ckH9OwTYOj3/KvDVMuzDevYJ8PfAvPR8FvCzAvu3K/Ca9Hw08Icq/ZsGXNyK9129vzdgBvArQMDrgOtb2NdhwP3A+FbuR+BNwGuA23LTvgacnJ6fXO3/CbATsCL9u2N6vmMrf/9FPTquIow2vH1b2u6RwE+bva0mqed2SE0REZfHc3cZuo7smqEyGMotopouItZExI3p+XqyM/P6vPtGSc0EzonMdWTXGO/aor68FbgnIu5t0fYBiIirgYcqJuffa7U+1w4FFkfEQxHxMLCY7F6dHa/jgrAP1W7ZM6TbtzXQG4G1EXF3jfkBXC7pBmW3jCvanDTsdFaNIZV69m0RjierDqopeh/Ws0+2uEUU0HuLqEKlIdn9gOurzH69pJsl/UrS3oV2LNPf760s7z3Iqvpaf8y2ej/uEhFr0vP7gV2qtCnTvixU2e41WheV5PZt9aizr0fRdzV4YET0SNoZWCzpzvRXX9P7CPwH8CWyD6QvkQ3hHt+obdejnn0o6bPAJuDHNVbT1H3YriRtD/wC+IeIeKxi9o1kw3yPp2PDF5JdnFyktvi9pXMJDgc+XWV2GfbjsyIiJPm6uZy2DMJoo9u39ddXZbcGehewfx/r6En/PiDpArJht4Z9GNS7PyV9D7i4yqwB39JoIOrYh8cCbwfeGulgR5V1NHUfVjGUW0QVQtI2ZCH444g4v3J+PhgjYpGk70gaGxGF3Ui6jt9bU997A3AYcGNErK2cUYb9CKyVtGtErElDxw9UadNDdjyz1+5k51p0vG4aGi3r7dsOBu6MiNXVZkoaJWl073Oyk0MG/Q0aA1VxvOWdNbZdz+2QmtW/6cAngcMjYkONNq3Yh0O5RVTTpWOR3wfuiIhv1mjz4t5jlpKmkn1eFBnU9fzeFgJ/m84efR3waG4IsEg1R3VavR+T/Hut1ufaZcAhknZMh0AOSdM6X6vP1mn0g+zDejXwFLAWuCw377NkZ/LdBRyWm74I2C09n0gWkMuBnwPbNrm/PwBOrJi2G7Ao15+b02MZ2XBgkfvzR8CtwC1k/5l2rexjej2D7MzDe4rsY/o9rQJuSo95lf1r1T6stk+AuWShDbBdeo8tT++5iQXutwPJhrtvye27GcCJve9HsltcLUv77Trgrwt+71X9vVX0UWTf7nVPep9OKbKPqQ+jyILthblpLduPZIG8Bng6fRaeQHbs+b+Bu4FfAzultlOA/8wte3x6Py4Hjit6X7bq4VusmZlZV+umoVEzM7PncRCamVlXcxCamVlXcxCamVlXcxCamVlXcxCaFUzSpZIekVTt5gRmVjAHoVnxvg4c3epOmFnGQWjWJJJem25Wvl26S8oySa+KiP8G1re6f2aWact7jZq1g4hYImkh8GVgBHBuRBR2ezwzq4+D0Ky55pLdd/RJ4GMt7ouZVeGhUbPmGgNsT/Yt8Nu1uC9mVoWD0Ky5vgt8nux7Er/a4r6YWRUeGjVrEkl/CzwdET+RNAy4VtJbgC8CrwC2l7QaOCEiuuPrbsxKyN8+YWZmXc1Do2Zm1tUchGZm1tUchGZm1tUchGZm1tUchGZm1tUchGZm1tUchGZm1tX+D6f7sNJGqAFeAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 1152x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plotModel(predict_hidden, t='fcnn separation with hidden layer')\n",
    "plt.scatter(x1,x2,c=\"black\",s=50)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercise \n",
    "Play around with the values for x1 and x2 and check if the position at the decision boundary\n",
    "matches the predicted probability\n",
    "How does the decision boundary look?  \n",
    "\n",
    "Add a second hidden Layer, with 8 nodes. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "### Your code here ###\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Keras \n",
    "\n",
    "**Look at this part after the introduction of Keras.**\n",
    "\n",
    "\n",
    "We now do the same as above using Keras."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Forwad pass in keras"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "dense (Dense)                (None, 1)                 3         \n",
      "=================================================================\n",
      "Total params: 3\n",
      "Trainable params: 3\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model = Sequential()                                        # starts the definition of the network\n",
    "model.add(Dense(1, batch_input_shape=(None, 2),             # adds a new layer to the network with a single neuron  \n",
    "                activation='sigmoid'))                      # The input is a tensor of size (batch_size, 2), \n",
    "                                                            # since we don’t specify the Batch Size now, we use None as a placeholder\n",
    "                                                            # chooses the activation function ‘sigmoid’\n",
    "\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.set_weights([W,np.array([b])])                        ## set the weights of the model to w1 w2 and b"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(100, 1, 1)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.PathCollection at 0x7f34b6e48208>"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAcIAAAEWCAYAAAD1t5d8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3de7wcdX3/8dc7CYGUayCK3Hq4WwIKhhiwSAqCEGIBpQqhtULF5keBVmutYrWURm21VlErirFSL6AQL9igUUAJRqQRQhog4SIhEkmI0AACEQSSfH5/zPeEOXt29+w5Z3dndvf9fDz2cWZnvjPz2dk5+9nPdy6riMDMzKxXjSk6ADMzsyI5EZqZWU9zIjQzs57mRGhmZj3NidDMzHqaE6GZmfU0J8IRkPRyScskPS3pb4qOp+wkrZB0TFnXL+kmSe9oY0j5dX9Z0ofrTN8gad8a086WdHOdeVvyuiRdLOmKZi+3VST9maTr60w/RtKaYSyvsP3FWmNc0QF0qPcCCyPisKIDKRtJXwbWRMQH+8dFxMHFRTRw/ZIuBvaPiLe2er0p+V4REXuOdBkRsV3zIupNEXElcGX/c0kBHBARK4uLysrEFeHI9AErig6i3ST5i5MVohv3vW58TZ3KiXCYJN0IHAt8NnVbHShpgqRPSFot6UlJN6dxe0sKSWdJ+pWk9ZI+kFvWxZLmSfpq6mZdIWlqjfVK0iWSHpX0lKS7JB2Spm0t6d/TOh6RdJmkCWnaMZLWSPqHtP4HJf1ZbrlvkPS/aZkPpYqpf1p//OdI+hVwYxr/TUm/Tq91kaSD0/jZwJ8B703b5to0/kFJx+di/ZSkh9PjU5K2roj179LrXCfpL2psj2Ml3ZV7foOk23LPfyrpjfn1S5oB/ANwRorvjtwi+yT9LL0P10ualFvWKem9+U3qFjsoNy0k7Z97/mVJH5a0LfADYPe0rg2Sdq/2WoCJkr6f1v1zSftVW76kXSTNT+/VrcB++YVIer2ke9P78llAFdPfLukeSU9Iuk5SX8V6zpV0f3qdl0oaMH8tdfaHV6f9cWyu7Wn9213SGEkXSnpA0mPpf2HnNK3qvlex3p9I+pM0fFRq/4b0/DhJy9Lwli5kSYvS7Hek9+SM3PKG3O+qxLCbpDsl/X16vqOkL6VlrE37wthcHD9T9n/8GHCxpP0k3Zhe/3pJV0raKbf896XlPC3pPknHNRKXDVNE+DHMB3AT8I7c80vTuD2AscAfAlsDewMBfBGYABwKPAcclOa7GPgdMDPN96/A4hrrPBG4HdiJ7APuIGC3NO0SYD6wM7A9cC3wr2naMcBG4JMppj8Cfgu8PDf9FWRfil4JPAK8MU3rj/+rwLbAhDT+7Wk9WwOfApbl4vwy8OGK2B8Ejk/Dc4DFwEuBlwC3AB+qiHUOsFXaLs8AE6tsjwlp201KbR8B1qa4JgDPArtUWf/FZN2Vle/nA8CBad6bgI+maQem7fX6tJ73AiuB8Wl6kHW1Dnr96fWsGWJf+jLwGDCN7FDFlcBVuelblg9cBcxL78Uh6fXenKZNAp4G3pzi/Nu0Ld+Rpp+a4j4oreeDwC0V6/ke2f71+8D/ATNqxDxgGw6xP9wNnJR7fg3wd2n4nWlf2DPN+wXgG/X2vYo45gD/kYb/Ib2HH8tN+3QaPrt/O9V4z46hwf0u//8P7AP8Aphd8fq+kGJ+KXAr8P9ycWwE/jq9BxOA/cn2ra3J/h8WAZ9K7V8OPATsntsm+xX9+deNj8ID6MQHuURIlkCeBQ6t0q7/n3nP3LhbgVlp+GLgR7lpk4Fna6zzdemf7khgTG68yD6o98uNew3wyzTc/0++bW76POAfa6znU8AlFfHvW2db7JTa7Jief5n6ifABYGZu2onAg7lYnwXG5aY/ChxZY90/BU5L2+T69LpmkFXsd9ZY/8VUT4QfzD0/D/hhGv5HYF5u2hiyBHRMet6MRPifueczgXtzz4Psw3Is8ALwB7lp/8KLifBt5L5Epf1iDS/upz8Azql4Hc8Afbn1vLZiH7mwRsyDtmGd/eF9wJVpeOe0zv4vcPcAx+Xm3S29xnE0tu8d1/8+Az8kS06L0/OfAKel4bMZOhEOZ7+7ieyL5YPAmbnxu5J90Z2QG3cm2fkE/XH8aoj94Y3A/6bh/VMcxwNb1ZvPj9E93Ec9epOAbcg+4Gv5dW74GWC7OtO2kTQuIjbmFxARN6burkvJuvG+A7wnrfv3gNtzPVki++Ds90RE/Db3fDWwO4CkI4CPklUY48m+mX6zIv6Htiw46+b5CPAWsm+wm9OkScCTNV5/3u5p/YNiSR6reO2V2yvvJ6Rkk4afIKt4n0vPh6PWezQg3ojYLOkhsuq/WertH/1eQpYgHsqNy2/H3fPTIiJSnP36gE9L+kRunMheR/9yGoljgAb2hyuAe1JX8enATyNiXS6mayRtzi1yE1lC6Zd/DZX+BzhQ0q7AYcApwD+nbu1pZNVVo4az30F2CGAl8K3cuD6yinJd7n9xTMVrGPB6UuyfBo4mq6rHkO3HRMRKSe8i++JxsKTrgHdHxMPDeF3WAB8jHL31ZF10+w3VcLQi4jMRcThZ5Xgg8Pdp/c8CB0fETumxYww823Bi+iDq9/tA/z/T18m6VfeKiB2By6g4tkT2Dbrfn5J1sx0P7Ej2zZ3cPPm21TxM9oFRLZbh6k+E09PwT8gS4R9ROxEOFV+lAfGm42Z7kVWFkH1g/l6u/ctGsa56/o+sst8rN+73c8Pr8tNycfZ7iKyLbqfcY0JE3DLKuOruDxGxlixhnQb8OfC1iphOqohpmzRPv5rbMCKeITtc8E5geUQ8T9bV/m7ggYhYP8rXVs/FZP97X88dA32I7EvYpNzr2SEGnjVd+Xr+JY17RUTsALyV3P9fRHw9Il5Ltg8G8LGWvJoe50Q4ShGxGbgc+KSk3SWNlfQapRNAmiWdeHCEpK3IukJ/B2xO6/8icImkl6a2e0g6sWIR/yxpvKSjgT/mxapve+DxiPidpGlkH2z1bE/2z/4YWQL4l4rpjwBVr3tLvgF8UNJL0jf3i8iqhpG4hew4yjTg1ohYQfaBcQS1q4FHgL0lNbrvzwPekE6+2Ar4O7LX359AlgF/mt73GWRJOL+uXSTtOJwXVU1EbAK+Q3aCxe9JmgyclWvyfbKq4TRlZyP+DQOT8mXA+3Mnsuwo6S2jjYuh9wfIjvO9l+xY9HcqYvpI/0k7aZ84dZjr/wlwAS9+8bmp4nk1Q+2jjXiBrAreFviqpDGp0r0e+ISkHdLJQPtJ+qM6y9ke2AA8KWkPsi+3wJbrlV+XPkt+R/aFd3P1xdhoOBE2x3uAu4DbgMfJvrU1e9vuQJbwniDrynoM+Hia9j6ybprFkp4CfkSWIPr9Os33MNnJGOdGxL1p2nnAHElPkyWleUPE8dW0/rVkJ0Isrpj+JWCysjMPv1tl/g8DS4A7ybbZ0jRu2FJ371JgRaoGIKs+VkfEozVm6/8C8JikpQ2s4z6yb+n/QVYBnAycnFvfO9O435B1l303N++9ZIl/Vdoetc4abdQFZN11vyY7tvhfuXWtJ/tg/ijZvnEA8LPc9GvI9sur0j6yHDhplPHA0PsDZCeQ9AHXpCqu36fJeiOuT/vfYrIvMcPxE7JksqjG82ouBr6S3pPTh7m+LdI+cBpZV+7l6cvV28gOMdxN9j/3LbJjn7X8MzCFrBv5+wz8orA12fu5nuw9fynw/pHGa7Upopm9N1Y2asJF3WajJekBsq7ZHxUdi1klV4Rm1lLpWr+gyrWAZmVQaCKUdHm6gHV5btzOyi6Ovj/9nVhj3rNSm/slnVWtjZkVS9JNwOeB89PxbLOGVMsPFdMl6TOSVqabGkzJTduk7H7QyyTNH3JdRXaNSppOdqD4qxHRf5eUfyM7eeOjki4ku6j1fRXz7Ux2nGkq2TfN24HDI+KJtr4AMzNriWr5oWL6TLKbE8wkO7b86Yg4Ik3bEMO4T2+hFWFELCI7uSTvVOArafgrZBeYVjoRuCEiHk/J7wayC6nNzKwL1MgPeaeSJcmIiMXATpLqnZhUUxkvqN81d8Htrxl4cW2/PRh4YeoaalzgrOz+l7MBxm494fDtdu2r1oyXbD9+pPE23Q7jxw7dqB2eaeT6+PZ7/vF6/xvd5+nHny06BOtCqzc9tz4iXtLMZR6y1baxITY1uv4VZJeF9JsbEXOHsbpaeWAd2Y1JlpBde/vRiKh2BvsWZUyEW6S7Y4yq7zZt2LkAO/UdFEd/4PKq7c6bPtrLiprr2L4dig6BMUuvLTqEqlZfMdQVHt1l0dVVD5GYjco7nrhv9dCthmdDbOIfd9i70fX/LiKq/shAE/RFxFplv+V5o6S7IqLm3b/KeNboI/3lbfpb7XqwtQy8a8aevHinjxH53KJVo5nd2qjvrSO+9MvMukfNPNB/d6KIWEV2k4VX1VtQGRPhfF68Y8ZZwH9XaXMdcIKkiems0hPSuK6xcPVTRYfA5iknFx2CAdPPGHSegJllueJt6ezRI4EnI2Jdygv9P+02CTiK7AYHNRV9+cQ3yO4E8nJlv0N3DtmdFF4v6X6y+xd+NLWdKuk/ASLiceBDZHdyuQ2Yk8aNiqvCzuGq0Ky7VcsPyn4z89zUZAGwiuyuWl8ku0sWZD81tkTZ714uJDtGWDcRFnqMMCLOrDFp0I9PRsQSsp9Z6X9+Odk9PrvWwtVPFX6scPOUk0t7rLCXTD/jEB8rtJ5SJz/0Tw/g/CrjbyG7r23Dytg1aiVT1i5SV4Vm1gxOhBXK1j1ahmOFZmbdzInQGuKqsHg+acasNZwIq3BVaGbWO5wIrWGuCovnqtCs+ZwIa3BVaGbWG5wIbVhcFRbPVaFZczkR1uGq0Mys+zkR2rC5KjSzbuJEOARXhVZG7h41ax4nQhsRV4Vm1i2cCBvgqtDKyFWhWXM4EdqIuSosnpOh2eg5ETbIVaGZWXdyIrRRcVVYPFeFZqPjRDgMrgrNzLqPE6GNmqvC4rkqNBs5J8JhKltVaPU5GZrZUEqZCCW9XNKy3OMpSe+qaHOMpCdzbS4qKt4ilaV7tKxVoZnZUEqZCCPivog4LCIOAw4HngGuqdL0p/3tImJOu+JzVdhZXBWaWT2lTIQVjgMeiIjVRQdSVq4KzcxGrhMS4SzgGzWmvUbSHZJ+IOngdgblqtDKylWh2fCUOhFKGg+cAnyzyuSlQF9EHAr8B/DdGsuYLWmJpCXPb3iidcEWzFVhfb3UPWrWDSTNkHSfpJWSLqwyvU/SjyXdKekmSXvmpp0l6f70OGuodZU6EQInAUsj4pHKCRHxVERsSMMLgK0kTarSbm5ETI2IqeO3m9jU4FwVWlm5KrROJmkscClZDpgMnClpckWzfwe+GhGvBOYA/5rm3Rn4J+AIYBrwT5LqfviXPRGeSY1uUUkvk6Q0PI3stTzWxthKx1Vhfa4KzTrGNGBlRKyKiOeBq4BTK9pMBm5Mwwtz008EboiIxyPiCeAGYEa9lZU2EUraFng98J3cuHMlnZuevhlYLukO4DPArIiIdsfpqtDKylWhldyk/sNW6TE7N20P4KHc8zVpXN4dwGlp+E3A9pJ2aXDeAcaNJPp2iIjfArtUjLssN/xZ4LPtjqvsFq5+imP7dig6DDZPOZkxS68tOoxB+t56OquvmFd0GG0z/YxDWHT18qLDsB6x/c4TmP4nDX4Bu+y+9RExdRSrew/wWUlnA4uAtcCmkSyotBVhJ3FVaGbWVGuBvXLP90zjtoiIhyPitIh4FfCBNO43jcxbyYmwC/lYYX29dqzQXaTWgW4DDpC0T7p6YBYwP99A0iRJ/Tns/cDlafg64ARJE9NJMiekcTU5ETaJq0Izs+aIiI3ABWQJ7B5gXkSskDRH0imp2THAfZJ+AewKfCTN+zjwIbJkehswJ42ryYmwS7kqrM9VoVm5RcSCiDgwIvaLiP4kd1FEzE/D34qIA1Kbd0TEc7l5L4+I/dPjv4ZalxNhE7kqNDPrPE6EXawsVWFZuSo0M3AibDpXhYOVtXvUzAycCLueq8L6XBWamRNhC7gqHMxVoZmVlRNhi5QpGboqrM9VoVlvcyK0tnFVWB5OhmYvciJsIVeFZmbl50RobVXWqrDXukfBVaFZPyfCFnNVaGZWbk6E1nauCsvDVaGZE2FbuCo0MysvJ0IrhKvC8nBVaL3OibBNXBWamZWTE2GPKkMydFVYHq4KrZeVNhFKelDSXZKWSVpSZbokfUbSSkl3SppSRJzDUaaq0MzMMqVNhMmxEXFYREytMu0k4ID0mA18vq2RdQFXhbW5KjTrHWVPhPWcCnw1MouBnSTtVnRQQ3FVaGZWLmVOhAFcL+l2SbOrTN8DeCj3fE0aN4Ck2ZKWSFry/IYnWhTq8JQpGboqtDxXhdaLypwIXxsRU8i6QM+XNH0kC4mIuRExNSKmjt9uYnMjtK7Wi92jZr2otIkwItamv48C1wDTKpqsBfbKPd8zjesIrgoHclVYHq4KrdeUMhFK2lbS9v3DwAnA8opm84G3pbNHjwSejIh1bQ7VupyrQrPuV8pECOwK3CzpDuBW4PsR8UNJ50o6N7VZAKwCVgJfBM4rJtTu4KrQzMpE0gxJ96VL5C6sMv2SdHndMkm/kPSb3LRNuWnzh1rXuGYH3wwRsQo4tMr4y3LDAZzfzria7XOLVnHe9H2LDsOG0PfW01l9xbyiw2ir6WccwqKrKzthzNpD0ljgUuD1ZCdC3iZpfkTc3d8mIv421/6vgVflFvFsRBzW6PrKWhFaAVwVmllJTANWRsSqiHgeuIrskrlazgS+MdKVOREWrEwnzVhtvXis0CfNWItN6r+0LT3yl8k1dHkcgKQ+YB/gxtzobdIyF0t641CBlLJr1IqzcPVTHNu3Q6ExbJ5yMmOWXltoDGY2fON33rnxL42XfXt9jbuGDdcs4FsRsSk3ri8i1kraF7hR0l0R8UCtBbgiLAFXhZ3BVaFZ2wzn8rhZVHSL5i6/WwXcxMDjh4M4EdogPlZoZgW7DThA0j6SxpMlu0Fnf0r6A2Ai8D+5cRMlbZ2GJwFHAXdXzpvnRFgSrgo7g6tCs9aLiI3ABcB1wD3AvIhYIWmOpFNyTWcBV6WrCPodBCxJl98tBD6aP9u0Gh8jtKp8rNDMihQRC8iuF8+Pu6ji+cVV5rsFeMVw1uWKsERcFXYGV4Vm3cWJ0GryscLaejEZmnUrJ8KScVVoZtZeToRWl6vC2nqtKnT3qHUrJ0IzM+tpToQlVLbuUVeFtbkqNOt8ToRmZtbTnAhLylXhYK4Ky8FVoXUbJ0IzM+tpToQl5qpwMFeFZtZsToRmNmzuHrVuUrpEKGkvSQsl3S1phaR3VmlzjKQnJS1Lj4uqLasbuCoczFWhmTVT6RIhsBH4u4iYDBwJnC9pcpV2P42Iw9JjTntDNKuul5Khq0LrFqVLhBGxLiKWpuGnyX6CY49ioypW2arCMihrVdhKG55/gXn3/JKPL76Leff8kg3Pv1B0SGZdoXSJME/S3mS/LPzzKpNfI+kOST+QdHCdZcyWtETSkuc3PNGiSHtLGbpHy6wVVeGSdes5+msL+MjP7uCLy37BR352B0d/bQFL1q1v+rqGw1WhdYPSJkJJ2wHfBt4VEZWfvEuBvog4FPgP4Lu1lhMRcyNiakRMHb/dxNYF3GKuCgfrlapww/Mv8JcLfsZvX9jIsxs3AfDsxk389oWNW8ab2ciVMhFK2oosCV4ZEd+pnB4RT0XEhjS8ANhK0qShlvvM0880PdZe5aqwvmZWhQseWMPmAT/A/aLNESxY+VDT1mXWi0qXCCUJ+BJwT0R8skabl6V2SJpG9joea1+UxXBVOFgvVIWrn9ywpRKs9OzGTax+ckObIxrI3aPW6UqXCIGjgD8HXpe7PGKmpHMlnZvavBlYLukO4DPArIgaX5kr3P7jZa2Juge5KqyvWVVh347bMWHc2KrTJowbS9+O2zVlPWa9qnSJMCJujghFxCtzl0csiIjLIuKy1OazEXFwRBwaEUdGxC1Fx90urgoH6/aqcOZ+ezIm6wAZZIzEzP33anNEg7kqtE5WukRoncVVYX3NqAq3G78VX5x5FNtuNW5LZThh3Fi23WrclvFmNnI9mQg7vXvUVeFg3V4VTt1tEje/7Q188KhDmX3YgXzwqEO5+W1vYOpuQ54j1jauCq1T9WQitOZyVVhfs44VbrvVON5y0D6858hX8JaD9nElaF1N0gxJ90laKenCGm1Oz92O8+u58WdJuj89zhpqXT2bCF0Vdp9urwo7gatCawZJY4FLgZOAycCZlbfalHQA8H7gqIg4GHhXGr8z8E/AEcA04J8k1b2IvGcToTWXq8L6fA9Ss2GZBqyMiFUR8TxwFXBqRZu/BC6NiCcAIuLRNP5E4IaIeDxNuwGYUW9lPZ0IO70qtMFcFZp1hT2A/J0i1jD4ntMHAgdK+pmkxZJmDGPeAXyQoYN9btEqzpu+b9FhbLFw9VMc27dD0WGwecrJjFl6bdFhDNL31tNZfcW8osNoi+lnHMKiq5cXHYa12+/tOJwvo5MkLck9nxsRc4extnHAAcAxwJ7AIkmvGMb8AxZkZmbWbusjYmqNaWuB/AWye6ZxeWuAn0fEC8AvJf2CLDGuJUuO+XlvqhdIT3eNQud3j5btpJmyHCssaxepjxWaNeQ24ABJ+0gaD8wC5le0+S4p4aV7TR8IrAKuA06QNDGdJHNCGldTzydCM2sdJ0MbiYjYCFxAlsDuAeZFxApJcySdkppdBzwm6W5gIfD3EfFYRDwOfIgsmd4GzEnjanIixFVhs7kqrK+XqkKzkUq31jwwIvaLiI+kcRdFxPw0HBHx7oiYHBGviIircvNeHhH7p8d/DbUuJ0IzaylXhVZ2ToSJq8LmKktVWFauCs3Kw4mwi5QtGZZBWbtHe42rQiszJ8KcTq8Ky8ZVYX2uCs3KwYmwy7gqHMxVYTm4KrSyciKs4KqwuVwV1ueq0Kx4ToRdyFXhYK4Ky8FVoZVR3UQoaQdJ+1UZ/8rWhbRlHXV/i0rS1pKuTtN/LmnvVsdkI+OqsD5XhWbFqpkIJZ0O3At8O/3o4atzk7/cyqAa+S0q4BzgiYjYH7gE+Fiz1t8N3aOuCgdzVVgOrgqtbOpVhP8AHB4RhwF/AXxN0pvSNLU4rkZ+i+pU4Ctp+FvAcZJaHZeNkKvC+nqtKnQytDKplwjHRsQ6gIi4FTgW+KCkvwGixXE18ntSW9qk+9I9CexSuSBJsyUtkbRk83MbGg7AVWF3clVoZpXqJcKn88cHU1I8hqwSO7jFcTVNRMyNiKkRMXXM1tsVHU5Pc1Voea4KrSzqJcK/Asbkj81FxNNkP3n/jhbH1chvUW1pI2kcsCPwWDODcFXYncpaFfZa96hZWdRMhBFxR0TcD8yT9D5lJgCfBM5rcVyN/BbVfOCsNPxm4MaIaHWXrY2Sq0LLc1VoZdDIdYRHkFVet5AlqIeBo1oZVIO/RfUlYBdJK4F3A4MusWgGV4XdyVWhmfUb10CbF4BngQnANsAvI2JzS6Mi+y0qYEHFuItyw78D3tLqOKz5Fq5+imP7dig6DCuJ6WccwqKrlxcdhvWwRirC28gS4auBo8mu6ftmS6MqGVeF3clVoZlBY4nwnPSrwC9ExLqIOJXBx+vMhsXHCi3PxwqtSEMmwohYUmXc11oTTnm5KuxOrgrNzDfdtsK4KrQ8V4VWFCfCYXBV2J1cFZaHk6EVwYnQCuWq0MyK5kTYg1wVDuaqsDxcFRoM/VN8uXZ/IikkTU3P95b0rKRl6XHZUOtyIhymbugehXIlQ1eFZpbX4E/xIWl74J3AzysmPRARh6XHuUOtz4lwBLolGdpArgrLw1Vhz2vkp/gAPkT2W7S/G83KnAh7mKtCMyupIX+KT9IUYK+I+H6V+feR9L+SfiLp6KFW1sgt1qyK23+8jMOPO6zoMKzJNk85mTFLry06DMO3XutETz2/aThfaidJyl+nPjci5jYyo6QxZD8AcXaVyeuA34+IxyQdDnxX0sERUTMwV4Q9zlVhZ+jF7lHreuv7fys2PfJJcKif4tseOAS4SdKDwJHAfElTI+K5iHgMICJuBx4ADqwXiBPhKPhYYXcq67FCsx5S96f4IuLJiJgUEXtHxN7AYuCUiFgi6SXpZBsk7QscANT9xu9EaK4KO0QvVoU+aaY3NfhTfLVMB+6UtAz4FnBuRDxebwYfIxwlHyvsTj5WaFasoX6Kr2L8MbnhbwPfHs66XBEa4KqwU7gqNGs+J8Im8LFCM7PO5URoW7gqHKisJ824KjRrrlIlQkkfl3SvpDslXSNppxrtHpR0V7qP3KDfSyyCq0Izs85UqkQI3AAcEhGvBH4BvL9O22PTfeSmtie0oXVDMnRVOJCrwvJwVWitUqpEGBHXp9NmIbsuZM8i4zEzs+5XqkRY4e3AD2pMC+B6SbdLml1vIZJmS1oiacnm5zY0PchKrgqby1Vhba4KzZqj7dcRSvoR8LIqkz4QEf+d2nwA2AhcWWMxr42ItZJeCtwg6d6IWFStYbptz1yA8Tv3xahfgJmZdZW2V4QRcXxEHFLl0Z8Ezwb+GPiziKiauCJibfr7KHAN2U92lIarwuZyVVhbL1aFZs1Wqq5RSTOA95LdM+6ZGm22TT/GiKRtgRMA36LerEe4e9SarVSJEPgs2V3Fb0iXRlwGIGl3Sf232tkVuFnSHcCtwPcj4ofFhFtbN1SFZeKqsDZXhWajU6p7jUbE/jXGPwzMTMOrgEPbGVev+tyiVZw3fd+iwzAbxL9VaM1Utoqwq7gqbC5XhbW5KjQbOSfCFuv0ZFimk2bKwsmwHHys0JrFidA6ShmqQjPrLk6EbeCqsPu4KiwHV4XWDE6E1nFcFZpZMzkRtomrwu7jqrAcXBXaaDkRWkdyVWhmzeJE2EauCruPq0KzzudE2GadngzLxFWh9XP3qI2GE6ENi6vCwVwVmnU2J8ICuCpsHleF1s9VoY2UE6ENm6vCwVwVmjWXpBmS7pO0UtKFVaafK+mu9AMNN0uanJv2/jTffZJOHGpdToQFcVXYPK4KzbqLpLHApcBJwGTgzHyiS74eEa+IiNTB0qAAAA8cSURBVMOAfwM+meadDMwCDgZmAJ9Ly6vJidBGxFXhYK4Ki+fu0a4xDVgZEasi4nngKuDUfIOIyH8D3hbo/yH3U4GrIuK5iPglsJIhfry9VD/D1Gtu//EyDj/usKLD6AoLVz/FsX07FB2GWU/7v6efH86X5EmSluSez42IuWl4D+Ch3LQ1wBGVC5B0PvBuYDzwuty8iyvm3aNeIK4IbcRcFQ7mqrB4rgo7xvqImJp7zB16loEi4tKI2A94H/DBkQbiRGhdw8cKzbrGWmCv3PM907hargLeOMJ5nQiL5pNmuo+rwuK5Kux4twEHSNpH0niyk1/m5xtIOiD39A3A/Wl4PjBL0taS9gEOAG6tt7LSJUJJF0tam06JXSZpZo12dU+ttfYoW/eoq8L6eikZWueKiI3ABcB1wD3AvIhYIWmOpFNSswskrZC0jOw44Vlp3hXAPOBu4IfA+RGxqd76ynqyzCUR8e+1JuZOrX092YHQ2yTNj4i72xVgM/mkme6zecrJjFl6bdFh9LTpZxzCoquXFx2GjVBELAAWVIy7KDf8zjrzfgT4SKPrKl1F2KAhT6219nFV2FlcFZoNVNZEeIGkOyVdLmlilenVTq2tenqspNmSlkhasvm5Da2ItSl8rLD7lPVYYS/xsUJrRCGJUNKPJC2v8jgV+DywH3AYsA74xGjWFRFz+0/PHbP1dk2I3qpxVdhZXBWavaiQY4QRcXwj7SR9EfhelUnDPj3WrAg+Vlg8Hyu0oZSua1TSbrmnbwKq7cFDnlrbiTq9e9RVYWfpparQXaRWT+kSIfBv6Y7idwLHAn8LIGl3SQug9qm1RQVsVo+PFZqVW+kSYUT8ebqj+Csj4pSIWJfGPxwRM3PtFkTEgRGxXzpVtiu4KmwuV4X1uSo0K2EiNOtGrgrNysuJsIRcFTaXq8L6XBVar3MiNGsTV4Vm5eREaC3hqrCzuCq0XuZEWFKd3j1q1bkqLAcnQ8tzIrSWcVXYWXqpKjTLcyIsMVeF3clVYTm4KrR+ToTWUq4KO4urQutFToQl56qwO7kqLAdXhQZOhNYGrgrNrMycCDuAq8LuVNaqsNe6R10VmhOhtYWrQjMrKyfCDtENVWHZkmEZuCosB1eFvc2J0HqWq0IzAyfCjuKqsDu5KiwHV4W9y4nQepqrQstzMiwPSTMk3SdppaQLq0yfLmmppI2S3lwxbZOkZekxf6h1ORF2GFeF3clVodmLJI0FLgVOAiYDZ0qaXNHsV8DZwNerLOLZiDgsPU4Zan1OhNbzXBVanqvCUpgGrIyIVRHxPHAVcGq+QUQ8GBF3AptHu7Jxo11AM0m6Gnh5eroT8JuIOKxKuweBp4FNwMaImNq2IEvg9h8v4/DjBm2WjvK5Ras4b/q+RYdRKpunnMyYpdcWHcYgfW89ndVXzCs6DOsAzzz9zHB6rSZJWpJ7Pjci5qbhPYCHctPWAEcMI5Rt0rI3Ah+NiO/Wa1yqRBgRZ/QPS/oE8GSd5sdGxPrWR2W9YOHqpzi2b4eiw7CSmH7GISy6ennRYXS79S0sYvoiYq2kfYEbJd0VEQ/UalzKrlFJAk4HvlF0LNY6PlY4mI8VmgGwFtgr93zPNK4hEbE2/V0F3AS8ql77UiZC4GjgkYi4v8b0AK6XdLuk2W2MqzS64aSZsvGxQsvzscJC3QYcIGkfSeOBWcCQZ38CSJooaes0PAk4Cri73jxtT4SSfiRpeZVH/kDomdSvBl8bEVPIzig6X9L0OuubLWmJpCWbn9vQpFdhzeKqcDBXhdbrImIjcAFwHXAPMC8iVkiaI+kUAEmvlrQGeAvwBUkr0uwHAUsk3QEsJDtGWDcRtv0YYUQcX2+6pHHAacDhdZbRX/Y+KukasjOMFtVoOxeYCzB+574YYdil1A0nzZSNjxVano8VFiciFgALKsZdlBu+jazLtHK+W4BXDGddZewaPR64NyLWVJsoaVtJ2/cPAycA3lM7mKtCKzN3kXa/MibCWVR0i0raXVL/N4NdgZtT2Xsr8P2I+GGbYywNHytsvjIcK3T3qFn7lOryCYCIOLvKuIeBmWl4FXBom8OyFvN1hWZWlDJWhDZMrgqbz1Vhbb1YFbp7tLs5EVpp+FihmRXBibBLdEtVWKZk6KqwNleF1k2cCLtItyRDM7N2ciK00nFVOJCrwvJwVdidnAi7jKtCM7PhcSK0UnJVOJCrwvJwVdh9nAi7kKtCM7PGORFaabkqHKisVWEvclXYXZwIu5SrQmuXXuwete7iRGil5qpwIFeF5eGqsHs4EXYxV4XWLq4KrZM5EVrpuSocyFWhWXM5EXY5V4XWLr1YFbp7tDs4EfYAJ8PmclVYWy8mQ+t8ToTWEcrUPWqW56qw8zkR9ghXhc3lqrA2V4XWaZwIrWO4KrSyclXY2ZwIe4irwuZyVVibq0LrJIUkQklvkbRC0mZJUyumvV/SSkn3STqxxvz7SPp5ane1pPHtidyK5qrQyspVYXNJmpHywEpJF1aZvnX6/F+Z8sHeuWlD5pG8oirC5cBpwKL8SEmTgVnAwcAM4HOSxlaZ/2PAJRGxP/AEcE5rw+0ergqby1Vhba4KbaTS5/6lwEnAZODMlB/yzgGeSHngErK8MJw8skUhiTAi7omI+6pMOhW4KiKei4hfAiuBafkGkgS8DvhWGvUV4I2tjNfKxVWhlZWrwqaZBqyMiFUR8TxwFVl+yDuV7PMfsnxwXMoPQ+aRSuOaGvro7QEszj1fk8bl7QL8JiI21mmzhaTZwOz09Ll1885b3qRYW2ESsL7VK/nevFHN3pYYh/K92pNKEd8QHOPolT0+KH+ML2/2Al944lfXrZt33qQGm28jaUnu+dyImJuG9wAeyk1bAxxRMf+WNhGxUdKTZPmhkTwyQMsSoaQfAS+rMukDEfHfrVpvpbRh56aYlkTE1CFmKUzZ44Pyx1j2+MAxNkPZ44Pyx1iRhJoiImY0e5nt0LJEGBHHj2C2tcBeued7pnF5jwE7SRqXqsJqbczMrHM1kgv626yRNA7YkSw/NDLvAGW7fGI+MCudDbQPcABwa75BRASwEHhzGnUW0LYK08zMWu424IB0hcB4spNf5le0mU/2+Q9ZPrgx5Ych80iloi6feJOkNcBrgO9Lug4gIlYA84C7gR8C50fEpjTPAkm7p0W8D3i3pJVkfcJfanDVc4duUqiyxwflj7Hs8YFjbIayxwflj7G08aXevguA64B7gHkRsULSHEmnpGZfAnZJeeDdwIVp3pp5pBZlCdTMzKw3la1r1MzMrK2cCM3MrKd1XSLspNu3peUvS48HJVW97Uuadldq1/RTnoeI8WJJa3NxzqzRru7tkFoY38cl3SvpTknXSNqpRru2b8PR3CKqDbHtJWmhpLvT/8s7q7Q5RtKTuff+onbFl4uh7vumzGfSNrxT0pQ2x/fy3PZZJukpSe+qaNPW7SjpckmPSlqeG7ezpBsk3Z/+Tqwx71mpzf2SzqrWpitFRFc9gIPILhS9CZiaGz8ZuAPYGtgHeAAYW2X+ecCsNHwZ8FdtivsTwEU1pj0ITCpoe14MvGeINmPT9twXGJ+28+Q2xXcCMC4Nfwz4WBm2YSPbBDgPuCwNzwKubmN8uwFT0vD2wC+qxHcM8L0i9rtG3zdgJvADQMCRwM8LjHUs8Gugr8jtCEwHpgDLc+P+DbgwDV9Y7f8E2BlYlf5OTMMTi3z/2/XouoowOvD2bWm9pwPfaPW6WqSR2yG1RERcHy/eZWgx2TVDZTCaW0S1XESsi4ilafhpsjPz6t59o6ROBb4amcVk1xjvVlAsxwEPRMTqgtYPQEQsAh6vGJ3f12p9rp0I3BARj0fEE8ANZPfq7HpdlwjrqHbLnlHdvq2JjgYeiYj7a0wP4HpJtyu7ZVy7XZC6nS6v0aXSyLZth7eTVQfVtHsbNrJNBtwiCui/RVRbpS7ZVwE/rzL5NZLukPQDSQe3NbDMUO9bWfY9yKr6Wl9mi96Ou0bEujT8a2DXKm3KtC3bqmz3Gm2ISnL7tkY0GOuZ1K8GXxsRayW9FLhB0r3pW1/LYwQ+D3yI7APpQ2RduG9v1rob0cg2lPQBYCNwZY3FtHQbdipJ2wHfBt4VEZU/pbGUrJtvQzo2/F2yi5PbqSPet3QuwSnA+6tMLsN23CIiQpKvm8vpyEQYHXT7tqFiVXZroNOAw+ssY236+6ika8i63Zr2YdDo9pT0Rarf73rYtzQajga24dnAHwPHRTrYUWUZLd2GVYzmFlFtIWkrsiR4ZUR8p3J6PjFGxAJJn5M0KSLadiPpBt63lu57w3ASsDQiHqmcUIbtCDwiabeIWJe6jh+t0mYt2fHMfnuSnWvR9Xqpa7Sst287Hrg3ItZUmyhpW0nb9w+TnRzStl/QqDje8qYa627kdkitim8G8F7glIh4pkabIrbhaG4R1XLpWOSXgHsi4pM12rys/5ilpGlknxftTNSNvG/zgbels0ePBJ7MdQG2U81enaK3Y5Lf12p9rl0HnCBpYjoEckIa1/2KPlun2Q+yD+s1wHPAI8B1uWkfIDuT7z7gpNz4BcDuaXhfsgS5EvgmsHWL4/0ycG7FuN2BBbl47kiPFWTdge3cnl8D7gLuJPtn2q0yxvR8JtmZhw+0M8b0Pj0ELEuPyyrjK2obVtsmwByypA2wTdrHVqZ9bt82brfXknV335nbdjOBc/v3R7JbXK1I220x8Idt3veqvm8VMYrsB1wfSPvp1HbGmGLYliyx7ZgbV9h2JEvI64AX0mfhOWTHnn8M3A/8CNg5tZ0K/Gdu3ren/XEl8Bft3pZFPXyLNTMz62m91DVqZmY2iBOhmZn1NCdCMzPraU6EZmbW05wIzcyspzkRmrWZpB9K+o2kajcnMLM2cyI0a7+PA39edBBmlnEiNGsRSa9ONyvfJt0lZYWkQyLix8DTRcdnZpmOvNeoWSeIiNskzQc+DEwAroiItt0ez8wa40Ro1lpzyO47+jvgbwqOxcyqcNeoWWvtAmxH9ivw2xQci5lV4URo1lpfAP6R7HcSP1ZwLGZWhbtGzVpE0tuAFyLi65LGArdIeh3wz8AfANtJWgOcExG98XM3ZiXkX58wM7Oe5q5RMzPraU6EZmbW05wIzcyspzkRmplZT3MiNDOznuZEaGZmPc2J0MzMetr/Bw3W6shX9XnsAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 1152x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plotting the decision boundary\n",
    "plotModel(model.predict, 'fcnn separation without hidden layer with keras') \n",
    "plt.scatter(X[0][0],X[0][1],c=\"black\",s=50)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Forwad pass with hidden layer (keras)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential_1\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "dense_1 (Dense)              (None, 8)                 24        \n",
      "_________________________________________________________________\n",
      "dense_2 (Dense)              (None, 1)                 9         \n",
      "=================================================================\n",
      "Total params: 33\n",
      "Trainable params: 33\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model = Sequential()                                        \n",
    "model.add(Dense(8, batch_input_shape=(None, 2),activation='sigmoid'))                      \n",
    "model.add(Dense(1, activation='sigmoid'))                      \n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.set_weights([W1,b1,W2,b2]) ## set the weights of the model to W1, b1, W2 and b2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(100, 1, 1)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.PathCollection at 0x7f34b438bf60>"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAcIAAAEWCAYAAAD1t5d8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nO3de7xVdZ3/8ddbEAWkVEhTNITECmvSJKxfllimyJhYU4ZdRs2GrPhl0zSlXcyhmrGaMuenRVR2szIqL1Sk0piaUyroeMNLImKAiOEVBU3k8/tjfQ8utnvvs/c5+7L23u/n47EfZ++1vmut71n7nPM+n++6bEUEZmZmvWqbdnfAzMysnRyEZmbW0xyEZmbW0xyEZmbW0xyEZmbW0xyEZmbW0xyEBSXpJZJulLRe0kfa3Z+ik7RU0tSibl/SFZLeX+O6pkpaVWX+XEmfrTI/JO1dYd7xkq6upR/1kLRX2u7QRq97AH1ZIenQdvfDOkfbf2itok8Av4+I/drdkaKR9H1gVUR8pm9aROzbvh5tvX1JpwN7R8R7mrStk5qxXrNe5YqwuMYBS9vdiVYrQkVhnUGZpv0N889i73AQFpCky4FDgLMlPS5pH0nDJX1V0r2SHpV0dZrWNyR1nKS/SFon6dO5dZ0uab6kH6Zh1qWSJlfYriSdKekBSY9JukXSy9O87ST9Z9rG2jQ8NzzNmypplaRPpe2vkPTu3Hr/XtL/pnWuTBVT37y+/p8o6S/A5Wn6zyXdn77XqyTtm6bPAt4NfCLtm1+l6VuGw1Jfvy7pvvT4uqTtSvr6L+n7XCPphAr74xBJt+ReL5K0OPf6D5KOzm9f0jTgU8A7U/9uyq1ynKT/Se/DZZLG9PNzULaPkr4v6Qu51/+a2twn6X0l6xgtaUHa99cBLy6Z/9L0fT0k6U5Jx5Rs5xxJv0l9vlbSVstX6fsJkm5Pyy2X9IHcvFslvSX3etv0c7N/ev0aSX+U9Iikm5QbclY2xPxFSf8DbAAm9NOPl0m6R9Kx6fXukn4p6a9p+kdybU+X9AtJ50l6DDhe0hRJf0p9WSPpbEnDUnupwu+LdZiI8KOAD+AK4P251+ekaWOBIcD/AbYD9gIC+DYwHHgl8BTwsrTc6cCTwPS03H8A11TY5uHA9cCOgICXAbuleWcCC4CdgVHAr4D/SPOmApuAr6U+HQw8AbwkN/8VZP94/R2wFjg6zevr/w+BkcDwNP19aTvbAV8Hbsz18/vAF0r6vgI4ND2fA1wD7AK8APgj8PmSvs4Btk37ZQOwU5n9MTztuzGp7VpgderXcGAjMLrM9k8Hzivzft4N7JOWvQI4o8L7ULWP+e8fmJb69fK0/36S9ufeaf75wPw07+Wp/1eneSOBlcAJZIdJ9gfWAZNy23kQmJLm/xg4v0Kf+97Hoen135OFrsh+HjYAr0rzPgH8LLfsDOCW9Hxs2uZ0sp+XN6fXL8jtx78A+6Y+bVumLyuAQ4FXpbZHpunbkP18nwYMIwvR5cDhufftaeDo1HY4cADwmrStvYDbgY/29/viR2c92t4BPyq8MbkgTL+UG4FXlmnX9wdoj9y064CZ6fnpwO9y8yYBGyts843An9Mv/ja56SILthfnpr0WuCc9n0r2h3tkbv584LMVtvN14MyS/k+osi92TG2en15/n+pBeDcwPTfvcGBFrq8bSX+w07QHgNdU2PYfgLelfXJZ+r6mkVXsN1fY/umUD8LP5F5/CLikwjar9pGtg/BccoFKFrQB7E32j8/TwEtz8/+dZ4PwncAfSrb9LeBzue18JzdvOnBHhT73vY9DK8y/CDg5Pd8dWA88L73+BfCJ9PyTwI9Klr0UOC63H+f087uzAvg3YBUwNTf9QOAvJW1PBb6Xe9+u6mfdHwUurPb74kfnPTwG3hnGANuT/YGv5P7c8w3ADlXmbS9paERsyq8gIi6XdDZZ9TlO0gXAx9O2RwDXS+prLrI/tH0ejogncq/vJfuDh6QDgTPIKpJhZFXez0v6v3LLiqUhwBeBd5BVdJvTrDHAoxW+/7zd0/af05fkwZLvvXR/5V1JFkyr0vOHySqcp9LrelR7j0rV2sfdyaqSPvnv+wVklczKCvPHAQdKeiQ3bSjwowH2eQtJRwCfIwvmbch+fm4BiIj70tDmP0i6EDgCODnXp3fkh07JquLf517nv59KTgKujIgrctPGAbuXfL9DyP7ZKbtuSfuQjXRMTt/DUNL+rvT7EhGP1dA/KxAfI+wM68iG6Go6PjMYEfFfEXEAWeW4D/CvafsbgX0jYsf0eH5E5P8o7iRpZO71i4D70vOfkA2r7hkRzwfmkgXpVpvOPX8X2XDZocDzyaoNcsv095Ep95H90SvXl3r1BeEb0vMryYLwYCoHYSs/0mUNsGfu9Ytyz/9KVqlXmr+SLCx2zD12iIgPDqZD6XjsL4H/BHaNiB2BhWz9nv8AeA/ZPzt/iojVuT79qKRPIyPijNyytezfk4AXSTozN20l2ShGft2jImJ6lXV/E7gDmBgRzyM7/rvl+6jw+2IdxkHYASJiM9kQ2NfSwf4hkl7bdwJIo0h6taQDJW1LNhT6JLA5bf/bwJmSdkltx0o6vGQV/yZpmKTXA0fybNU3CngoIp6UNIUs6KoZRVZxPUj2X/i/l8xfS/WTJH4KfEbSC9IJKacB5/WzzUr+CLyE7DjZdRGxlFRJAVdVWGYtsJeaeEZjznyykzomSRpBVoUBEBHPABcAp0saIWkScFxu2V8D+0h6bzphZdv0M/CyQfapr+r/K7ApVYeHlbS5iOwY3slkx4f7nAe8RdLh6ed8e2UnOO1RZx/Wkw1hv0FSX4heB6yX9EllJ5oNkfRySa+usp5RwGPA45JeCmz5J6HS70ud/bQCcBB2jo+TDS0tBh4CvkTj37/nkQXew2RDaA8CX0nzPgksA65JZ9T9jiwg+tyflruP7KSKkyLijjTvQ8AcSevJQml+P/34Ydr+auA2shNf8r4LTEpn8l1UZvkvAEuAm8n22Q1pWt3ScO8NwNKI+Fua/Cfg3oh4oMJiff8APCjphoFst47+/ZbsmOvlZO/P5SVNZpMNZ95Pdszve7ll15MF1Eyy9+1+sp+rQf2Dldb7EbL3+WGyf3wWlLTZSFY1jicL677pK8lGAz5FFqQryaqsun/WI+IRspNtjpD0+fSPwZHAfsA9ZCMd3yEbdajk46n/68l+N36Wm1ft98U6iCL8wbw2OOn09vMiot7/2q2HSToN2CeadOMBs1r5ZBkzazlJOwMnAu9td1/M2jo0KuncdDHqrblpOyu7wPeu9HWnCssel9rcJem4cm3MrHgk/RPZkOdvI6LScVbrcZKmKbvJwzJJp5SZP07Sf0u6Od1oYY/cvLryoa1Do5LeADwO/DAi+u5g8mWyEyvOSN/8ThHxyZLldiY7BjSZ7Cyv64EDIuLhln4DZmbWcOkSqj+THeNdRXZuxLERcVuuzc+BX0fEDyS9ETghIt47kHxoa0WY/ht8qGTyDLJTq0lfjy6z6OHAooh4KH1zi8jOEDMzs843BVgWEcvTSWrnk2VD3iSePTns97n5dedDEY8R7hoRa9Lz+4Fdy7QZy9YXvq5K055D2b0pZwFo6HYHDH3ebg3sqlln2XHHEe3uQkfZZYdhTd/G9vFUw9b19IPram57831/XRcRL2jYxoFXDtsh1sczNbW9Z9OTS8kuOekzLyLmpefl/sYfWLKKm8ju+nQW8FZglKTRFZYtmw99ihiEW0RESBrU2G3asfMAho0eH7tO/1w/S5h1pxlHv6rdXegosw/aq+nbmLhxecPWtfa8b9XVfuxn597bf6v6rI9n+MKO42tq++51tz8ZEWU/AKBGHyf7YILjya7pXQ3UlsIlihiEayXtFhFrJO1Gdo/FUqvJ7vbRZw+yexCamQ1as0OwnQHYIVaz9R2R9kjTtoiI+8gqQiTtAPxDRDwiqe58KOIF9Qt49u4XxwEXl2lzKXCYpJ3SWaWHpWlmVoarwdrMPmgvh2AxLAYmShqfPvZqJiU3ZZA0Jnf3plPJ7r4FA8iHtlaEkn5KltxjJK0iuz3UGcB8SSeS3a3hmNR2MtndSt4fEQ9J+jzZzoLsbvSlJ92YmdXMAVgcEbFJ0myyABsCnBsRSyXNAZZExAKy7PiPdPjsKuDDadm686Gn7izjY4TWi1wN9q8XQ3DsZ+deP8hjdM8xYdvhUccxwoZvf6CKeIzQzKwlejEA7bmKeIzQzKzpHILWxxWhWRfzsGh5nRKCDsDWcBCaWc9wAFo5Hho161KuBrfmELRKXBGaWddrZgg6ADufK0KzLuRq8FkOQeuPK0Iz60qdMBTajgC84qwrW77NonMQmnUZV4OuAitxCJbnIDSzrlL0EHQAFo+D0My6QtEDEFofgg7A2jgIzbpIrw6LFj0EXQUWm4PQzDqWT4h5Lgdg/RyEZl2i16pBV4FbcwAOnK8jNLOO4xDcmkNwcFwRmnWBXqkGPRS6NQdgYzgIzawjuArcmkOwcRyEZlZ4DsFnOQAbr5BBKOklwM9ykyYAp0XE13NtpgIXA/ekSRdExJyWddKsILp5WNQB+CwHYPMUMggj4k5gPwBJQ4DVwIVlmv4hIo5sZd/MisQhODAOQcsrZBCWeBNwd0Tc2+6OmFlrNCsEHYBWTidcPjET+GmFea+VdJOk30rat5WdMmu3bq0GHYIZh2DrFLoilDQMOAo4tczsG4BxEfG4pOnARcDEMuuYBcwCGDJydBN7a9Y63RiCDsCMA7D1Ch2EwBHADRGxtnRGRDyWe75Q0jckjYmIdSXt5gHzAIaNHh/N7rCZ1c8h6ABsp6IH4bFUGBaV9EJgbUSEpClkw7wPtrJzZjZ4zQjBTgpAcAi2W2GDUNJI4M3AB3LTTgKIiLnA24EPStoEbARmRoQrPut63TQs2ush6AAshsIGYUQ8AYwumTY39/xs4OxW98vMBq/XAxAcgkVS2CA0s+fqhmqw10PQAVgbSdOAs4AhwHci4oyS+WcCh6SXI4BdImLHNO8Z4JY07y8RcVS1bTkIzaxlihiCrgKLJ91I5Ryyw2OrgMWSFkTEbX1tIuKfc+3/L7B/bhUbI2K/WrfnIDSzlmh0CLoK7GpTgGURsRxA0vnADOC2Cu2PBT430I05CM06RKcOi7oKdAgOwFhgZe71KuDAcg0ljQPGA5fnJm8vaQmwCTgjIi6qtjEHoZk1TdFC0AHYXKN2GcXUDx5cW+PP3j4mhVWfeem673rNBH4REc/kpo2LiNWSJgCXS7olIu6utAIHoZk1RdGGQj0MWjjrImJyhXmrgT1zr/dI08qZCXw4PyEiVqevyyVdQXb80EFo1sk6bVi0SCHoKrAjLQYmShpPFoAzgXeVNpL0UmAn4E+5aTsBGyLiKUljgNcBX662MQehmTVMkQIQXAV2qojYJGk2cCnZ5RPnRsRSSXOAJRGxIDWdCZxfcjOVlwHfkrSZ7I5jZ+TPNi3HQWhWcJ1SDRYpBF0Fdr6IWAgsLJl2Wsnr08ss90fgFfVsy0FoZoPWiyHoAOweDkIzG5RGhmAnDIU6ALuPg9CswIo8LOoq0LqFg9DM6tZrIegA7G4OQrOCKmo16KFQ6zYOQjOriatA61YOQjPrVy9VgQ7A3uMgNCugIg2LFiUEXQVaszgIzQqmG0PQVaAVmYPQrEAcgltzFWitUNgglLQCWA88A2wqvUu5JAFnAdOBDcDxEXFDq/tp1o0aEYKuAq1TFDYIk0MiYl2FeUcAE9PjQOCbVPjgRrNOUIRq0FWg9aKiB2E1M4AfpruOXyNpR0m7RcSadnfMrF4OwYyrQGuHIgdhAJdJCuBbZT65eCywMvd6VZq2VRBKmgXMAhgycnTzemvWwdo9FOoq0NqpyEF4UESslrQLsEjSHRFxVb0rSQE6D2DY6PHRT3Ozlmt3NdjtIegAtP4UNggjYnX6+oCkC4EpQD4IVwN75l7vkaaZdYxOD8EiByA4BK02hQxCSSOBbSJifXp+GDCnpNkCYLak88lOknnUxwfNatfNIegAtHoUMgiBXYELsyskGAr8JCIukXQSQETMJfvk4unAMrLLJ05oU1/NBqSd1WC7QtBVoBVRIYMwIpYDrywzfW7ueQAfbmW/zBqlU0PQVaB1o0IGoVk367UQdBVoRecgNOsB3ToU6gC0RnAQmrVQO6pBV4Fm1TkIzVrEIdgYDkBrNAehWZdqdQi6CrRO5SA0a4FWV4MDDUFXgdaLtml3B8y6nUNwcByC1myuCM2aqJtD0AFo3cIVoVmXcAiaDYwrQrMmaVU12E1DoQ5A6yNpGnAWMAT4TkScUabNMcDpZB/bd1NEvCtNPw74TGr2hYj4QbVtOQjNmqAbQ9BVoLWKpCHAOcCbyT5rdrGkBRFxW67NROBU4HUR8XD6yD4k7Qx8DphMFpDXp2UfrrQ9B6FZh+qWEHQAWhlTgGXpvtOkTxmaAdyWa/NPwDl9ARcRD6TphwOLIuKhtOwiYBrw00obcxCaNVgrqsGBhKCHQq3Zth09hl3f84HaGn927hhJS3JT5qUPUgcYC6zMzVtF9nF7efsASPofsuHT0yPikgrLjq3WFQehWYdpVQg6AK3J1kXE5EEsPxSYCEwl+2D2qyS9YqArMrMOUW8Iugq0DrUa2DP3eo80LW8VcG1EPA3cI+nPZMG4miwc88teUW1jDkKzBmrmsGinh6AD0OqwGJgoaTxZsM0E3lXS5iLgWOB7ksaQDZUuB+4G/l3STqndYWQn1VTkIDTrAK0IQVeBVhQRsUnSbOBSsuN/50bEUklzgCURsSDNO0zSbcAzwL9GxIMAkj5PFqYAc/pOnKnEQWjWIM2qBusJQVeB1i0iYiGwsGTaabnnAXwsPUqXPRc4t9ZtFS4IJe0J/BDYlewakHkRcVZJm6nAxcA9adIFETGnlf00a4Vmh6CrQLMCBiGwCfiXiLhB0iiyiyEX5S+kTP4QEUe2oX9mLdGpIegAtE5TuCCMiDXAmvR8vaTbya4BKQ1Cs8Jo9LBouRB84vH1LLz4Au5dvpxxEyYwfcbb2G/IXwe0foeg2bMKF4R5kvYC9geuLTP7tZJuAu4DPh4RSyusYxYwC2DIyNHN6ahZA5ULweuv/RMfePfb2bx5Mxs3bGD4iBF85bRT+NX3zuagV+9f87odgGbPVdhPn5C0A/BL4KMR8VjJ7BuAcRHxSuD/kZ1GW1ZEzIuIyRExeZvtRjWvw2YNUKkS/MC7384Tjz/Oxg0bANi4YQPrn9jAW06YzeNPbKhp3Q5Bs/IKGYSStiULwR9HxAWl8yPisYh4PD1fCGybriMxa7lGDYtWOia48OIL2Lx5c9l5myOY/+tL+123Q9CsssINjUoS8F3g9oj4WoU2LwTWRkRImkIW6A+2sJtmDVXtxJh7ly/fUgmWemLDRpbdu7LsPPBZoWa1KFwQAq8D3gvcIunGNO1TwIsAImIu8Hbgg5I2ARuBmemaErOWakQ12N/ZoeMmTGD4iBFlw3DkiOHsPW7PMku5CjSrVeGCMCKuBtRPm7OBs1vTI7PmqeUSiekz3sZXTjul7LxtJI458vCtprkKNKtPIY8RmvWCWq8T3G/IX/nV985m1MgRjBwxHMgqwVEjR/Cr753NDiNHbGnrEDSrX+EqQrNOMZhh0VpDsO9C+YNevT8rr13E/F9fyrJ7V7L3uD055sjDHYJmDeAgNGuxekOwzw4jR/C+d771Oe2aGYDgELTu5yA0a6GBhmAlrgLNBs/HCM0GYCDDog5Bs2JyEJq1gEPQrLgchGZ1qrcadAiaFZuD0KyJHIJmxecgNGsSh6BZZ/BZo2Z1qHVYtFNC0AFo5orQrGc5BM0yDkKzGnVTNegQNHuWg9CsxzgEzbbmIDSrQbdUgw5Bs+fyyTJm/WjUJ9C3kwPQrDJXhGZV1BOCja4GG8UhaFadg9CsgnZXgo0YFnUImvXPQWhWRrNuo9ZKDkGz2lQNQknPk/TiMtP/rnld2rKNaZLulLRM0ill5m8n6Wdp/rWS9mp2n6w3NLMSbNVJMg5B63T9ZUCu3T9ICkmT0+u9JG2UdGN6zO1vWxVPlpF0DPB14AFJ2wLHR8TiNPv7QNP+WkgaApwDvBlYBSyWtCAibss1OxF4OCL2ljQT+BLwzmb1yXpDMz9eqRUcgNYNaswAJI0CTgauLVnF3RGxX63bq1YRfgo4IK3sBOBHkvo+Hlu1bmCApgDLImJ5RPwNOB+YUdJmBvCD9PwXwJskNbtf1sWafUyw2SfJOASti9SSAQCfJyuCnhzMxqoF4ZCIWAMQEdcBhwCfkfQRIAaz0RqMBVbmXq9K08q2iYhNwKPA6NIVSZolaYmkJZufWt+k7lqnG2gINqMaHMiwqEPQOtCYvr/N6TErN6/fDJD0KmDPiPhNmXWPl/S/kq6U9Pr+OlLtOsL1kl4cEXcDRMQaSVOBi4B9+1txUUTEPGAewLDR45sd4NaBWnF2aDOrQYegFcWT2o67hk+otfm6iJg8kO1I2gb4GnB8mdlrgBdFxIOSDgAukrRvRDxWaX3VKsIPAttImtQ3ISLWA9OA9w+k83VYDeyZe71Hmla2jaShwPOBB5vcL+sygwnBIlSDDkHrUv1lwCjg5cAVklYArwEWSJocEU9FxIMAEXE9cDewT7WNVQzCiLgpIu4C5kv6pDLDyVL4Q/V/X3VZDEyUNF7SMGAmsKCkzQLguPT87cDlEeGKz2rWqusEm1UNOgSti1XNgIh4NCLGRMReEbEXcA1wVEQskfSCdLINkiYAE4Gqv4S1XEd4IFky/zF17j7gdfV/X7VLx/xmA5cCtwPzI2KppDmSjkrNvguMlrQM+BhQ8fRas1KDDcF2V4MOQetmNWZAJW8AbpZ0I9mJlCdFxEPVFqjlXqNPAxuB4cD2wD0RsbmG5QYlIhYCC0umnZZ7/iTwjmb3w7pPK+8Y0+rbqZl1i/4yoGT61NzzXwK/rGdbtVSEi8mC8NXA64FjJf28no2YFUUjQrDd1w26GjRrrFoqwhMjYkl6vgaYIem9TeyTWVO0+96h1TTjw3fNrDb9VoS5EMxP+1FzumPWHI0KwXqqQQ+LmnUG33Tbul6RK0HwSTJm7eYgtK7WyBB0NWjWnRyE1rWKXgnWy9WgWXM4CK0rtTME66kGfZKMWfs5CM1q0O5LJsyseRyE1nUaXQ06BM26m4PQukq3HRc0s+ZzEJpVUW816LNFzTqPg9C6hqtBMxsIB6FZBc0+NugzRs2KwUFoXaEIJ8h4WNSsMzkIreN5SNTMBsNBaFbCl0uY9RYHoXW0IgyJgodFzTqZg9DMzHpaLR/M2zKSvgK8BfgbcDdwQkQ8UqbdCmA98AywKSImt7KfVgxFqQbNrLMVrSJcBLw8Iv4O+DNwapW2h0TEfg5B60S+dMKsOAoVhBFxWURsSi+vAfZoZ3+suIpUDfr4oFlnK1QQlngf8NsK8wK4TNL1kmZVW4mkWZKWSFqy+an1De+ktV6RQtDMOl/LjxFK+h3wwjKzPh0RF6c2nwY2AT+usJqDImK1pF2ARZLuiIiryjWMiHnAPIBho8fHoL8BMzPrKi0Pwog4tNp8SccDRwJvioiywRURq9PXByRdCEwBygahdZeiVYMeFjXrfIUaGpU0DfgEcFREbKjQZqSkUX3PgcOAW1vXSzMz6yaFCkLgbGAU2XDnjZLmAkjaXdLC1GZX4GpJNwHXAb+JiEva011rpaJVg2bWHQp1HWFE7F1h+n3A9PR8OfDKVvbL2q+bQtCXTpgVS9EqQrOO4eODZs0jaZqkOyUtk3RKmfknSboljR5eLWlSbt6pabk7JR3e37YchFZ43VQNmln/JA0BzgGOACYBx+aDLvlJRLwiIvYDvgx8LS07CZgJ7AtMA76R1leRg9B6ikPQrCNMAZZFxPKI+BtwPjAj3yAiHsu9HEl2fTmp3fkR8VRE3AMsS+urqFDHCM1K+bMGzbrWGElLcq/npeu+AcYCK3PzVgEHlq5A0oeBjwHDgDfmlr2mZNmx1TriILTC8pDo1qaefDBXnHVlu7thVtEDj/+Ns69eUWvzdYO9V3REnAOcI+ldwGeA4wayHg+NmplZ0awG9sy93iNNq+R84OgBLusgtOKZcfSruroa3PU9H2h3F8yKbjEwUdJ4ScPITn5ZkG8gaWLu5d8Dd6XnC4CZkraTNB6YSHbNeUUOQisUHxOsburJB7e7C2ZNlz6FaDZwKXA7MD8ilkqaI+mo1Gy2pKWSbiQ7TnhcWnYpMB+4DbgE+HBEPFNtez5GaIXRrBAsUjVoZrWJiIXAwpJpp+Wen1xl2S8CX6x1W64IrRBcCZpZuzgIra2acTwwrxurQQ+PmjWWg9DaxlWgmRWBg9DaohUhWORq0GeOmhWHT5axlnIVaGZF44rQWqaVIVjkarARfJzQrHEchNYSDsHGcxiaNYaD0JrOw6HN4zA0GzwHoTVNsy+NKKeTqkGfMGNWDIULQkmnS1qdPnX4RknTK7Sr+unF1l6uAlvHVaHZ4BQuCJMzI2K/9FhYOrPGTy+2NmlXCHZSNdhoDkOzgStqEPan308vttZrx1CoPcthaDYwRQ3C2ZJulnSupJ3KzC/36cVlP4FY0ixJSyQt2fzU+mb01Wj/UGgvV4N5DkOz+rUlCCX9TtKtZR4zgG8CLwb2A9YAXx3MtiJiXkRMjojJ22w3qgG9t1LtDsFO1owTZhyGZvVpy51lIuLQWtpJ+jbw6zKz6v4EYmuOIoSgq0EzG4zCDY1K2i338q3ArWWa9fvpxdZcPh5YbK4KzWpXuCAEvizpFkk3A4cA/wwgaXdJC6Hypxe3q8O9pkgB6GqwMoehWW0Kd9PtiHhvhen3AdNzr5/z6cXWfA7BzjL15IO54qwr290Ns0IrYkVoBVWkELTauTI0q85BaP3y8cDm8q3WzNrLQWhVFTUAPSxaH1eFZpU5CK2iooZgN3JVaNY+DkIryyHYv7uGT2h3F+riqtCsPAehbaUTjgd267Coq0Kz9nAQ2hZFD0AbPFeFZs/lIDSgc0KwW6vBPq4KzVrPQWgdE4LWGK4KzbbmIOxhnXA8sBe5KjQDSdMk3SlpmaRTysx/g6QbJFU00ucAAAvjSURBVG2S9PaSec9IujE9+r0PtYOwR3ViAHb7sGgruSq0IpM0BDgHOAKYBBwraVJJs78AxwM/KbOKjRGxX3oc1d/2HIQ9qBNDsKg67RIKsw4xBVgWEcsj4m/A+cCMfIOIWBERNwObB7sxB2EP8VBo52jF8KirQiuwscDK3OtVaVqttpe0RNI1ko7ur3HhPn3CmqPTA9DDombF98gjG7j4ohtqbT5G0pLc63kRMa9BXRkXEaslTQAul3RLRNxdqbErwi7nKrBzuSq0LrcuIibnHvkQXA3smXu9R5pWk4hYnb4uB64A9q/W3kHYxbolAF0NmvWcxcBESeMlDQNmAv2e/QkgaSdJ26XnY4DXAbdVW8ZB2KW6JQR7natC60URsQmYDVwK3A7Mj4ilkuZIOgpA0qslrQLeAXxL0tK0+MuAJZJuAn4PnBERVYPQxwi7jAPQzLpBRCwEFpZMOy33fDHZkGnpcn8EXlHPtgpVEUr6We4iyBWSbqzQboWkW1K7JeXa9KJuDMFOGBZt9iUUrgrNmqtQFWFEvLPvuaSvAo9WaX5IRKxrfq86QzeGoJlZKxSqIuwjScAxwE/b3Zei81mhvcFVoVnzFDIIgdcDayPirgrzA7hM0vWSZrWwX4XS7QHYCcOiZtb5Wj40Kul3wAvLzPp0RFycnh9L9WrwoHSx5C7AIkl3RMRVFbY3C5gFMGTk6EH0vDi6PQCtfaaefDBXnHVlu7th1lItD8KIOLTafElDgbcBB1RZR9/Fkg9IupDsvnRlgzBdpDkPYNjo8THAbhdGr4Sgq8Hn2vU9H2Dted9qdzfMuk4Rh0YPBe6IiFXlZkoaKWlU33PgMODWFvavbXolBK29fKzQek0Rg3AmJcOiknaX1Hc9ya7A1eliyeuA30TEJS3uY0v12gkxnVgNtupTKPxZhWaNV6jLJwAi4vgy0+4Dpqfny4FXtrhbbdNLAWhm1g5FrAgt6cUQ7MRqsNV8KYVZYzkIC6oXQ9DMrB0chAXTa8cD81wNFourQusVDsIC6dUAtPr5pBmzxnEQFoRDsPO16sxRM2ssB2EBOAQ9LDoQPmnGrDEKd/lEL3EAmpm1nyvCNnEIPsvVoJm1k4OwDRyC1igeHjUbPA+NtpADsPvdNXwCEzcub3c3zKwOrghbxCFYnodFB8+XUpgNjoOwyXr5AnnrHh4etW7mIGwiB2B1rgbNrAgchE3iELRW8kkzZgPnk2UazAFYG1eDZlYUrggbyCFo0L5brfmkGbOBcRA2gE+IsV7h4VHrRg7CQXIA1s/DomZWJA7CQXAIWtF4eNSsfm0JQknvkLRU0mZJk0vmnSppmaQ7JR1eYfnxkq5N7X4maVhrep7xUOjAuRrsfB4etVaQNC3lwDJJp5SZv136+78s5cFeuXn95kheuyrCW4G3AVflJ0qaBMwE9gWmAd+QNKTM8l8CzoyIvYGHgROb291nOQDNzJor/d0/BzgCmAQcm/Ih70Tg4ZQDZ5LlQj05skVbgjAibo+IO8vMmgGcHxFPRcQ9wDJgSr6BJAFvBH6RJv0AOLqZ/d3SOYeg1aidH9Lr4VHrAlOAZRGxPCL+BpxPlg95M8j+/kOWB29K+dBvjpQq2nWEY4Frcq9XpWl5o4FHImJTlTZbSJoFzEovn1p13gm3DrRz55w30CVrNgZY1/StDM6g+nhOAztSQdfvwxYpeh+L3j8ofh9f0ugVPv3QiktXnXfCmBqbby9pSe71vIiYl56PBVbm5q0CDixZfkubiNgk6VGyfKglR7bStCCU9DvghWVmfToiLm7WdkulHTsv9WlJREzuZ5G2KXr/oPh9LHr/wH1shKL3D4rfx5IQaoiImNbodbZC04IwIg4dwGKrgT1zr/dI0/IeBHaUNDRVheXamJlZ56olC/rarJI0FHg+WT7UsuxWinb5xAJgZjobaDwwEbgu3yAiAvg98PY06TigZRWmmZk13WJgYrpCYBjZyS8LStosIPv7D1keXJ7yod8cKdWuyyfeKmkV8FrgN5IuBYiIpcB84DbgEuDDEfFMWmahpN3TKj4JfEzSMrIx4e/WuOl5/Tdpq6L3D4rfx6L3D9zHRih6/6D4fSxs/9Jo32zgUuB2YH5ELJU0R9JRqdl3gdEpBz4GnJKWrZgjlSgLUDMzs95UtKFRMzOzlnIQmplZT+u6IOyk27el9d+YHisk3Vih3QpJt6R2DT/luZ8+ni5pda6f0yu0q3o7pCb27yuS7pB0s6QLJe1YoV3L9+FgbhHVgr7tKen3km5Lvy8nl2kzVdKjuff+tFb1L9eHqu+bMv+V9uHNklp61wtJL8ntnxslPSbpoyVtWrofJZ0r6QFJt+am7SxpkaS70tedKix7XGpzl6TjyrXpShHRVQ/gZWQXil4BTM5NnwTcBGwHjAfuBoaUWX4+MDM9nwt8sEX9/ipwWoV5K4AxbdqfpwMf76fNkLQ/JwDD0n6e1KL+HQYMTc+/BHypCPuwln0CfAiYm57PBH7Wwv7tBrwqPR8F/LlM/6YCv27Hz12t7xswHfgtIOA1wLVt7OsQ4H5gXDv3I/AG4FXArblpXwZOSc9PKfd7AuwMLE9fd0rPd2rn+9+qR9dVhNGBt29L2z0G+Gmzt9UktdwOqSki4rJ49i5D15BdM1QEg7lFVNNFxJqIuCE9X092Zl7Vu28U1Azgh5G5huwa493a1Jc3AXdHxL1t2j4AEXEV8FDJ5PzPWqW/a4cDiyLioYh4GFhEdq/Ortd1QVhFuVv2DOr2bQ30emBtRNxVYX4Al0m6Xtkt41ptdhp2OrfCkEot+7YV3kdWHZTT6n1Yyz7Z6hZRQN8toloqDcnuD1xbZvZrJd0k6beS9m1pxzL9vW9F+dmDrKqv9M9su/fjrhGxJj2/H9i1TJsi7cuWKtq9Rmuigty+rRY19vVYqleDB0XEakm7AIsk3ZH+62t6H4FvAp8n+4P0ebIh3Pc1atu1qGUfSvo0sAn4cYXVNHUfdipJOwC/BD4aEY+VzL6BbJjv8XRs+CKyi5NbqSPet3QuwVHAqWVmF2E/bhERIcnXzeV0ZBBGB92+rb++Krs10NuAA6qsY3X6+oCkC8mG3Rr2x6DW/Snp28Cvy8yq+5ZG9ahhHx4PHAm8KdLBjjLraOo+LGMwt4hqCUnbkoXgjyPigtL5+WCMiIWSviFpTES07EbSNbxvTf3Zq8MRwA0RsbZ0RhH2I7BW0m4RsSYNHT9Qps1qsuOZffYgO9ei6/XS0GhRb992KHBHRKwqN1PSSEmj+p6TnRwy4E/QqFfJ8Za3Vth2LbdDalb/pgGfAI6KiA0V2rRjHw7mFlFNl45Ffhe4PSK+VqHNC/uOWUqaQvb3opVBXcv7tgD4x3T26GuAR3NDgK1UcVSn3fsxyf+sVfq7dilwmKSd0iGQw9K07tfus3Ua/SD7Y70KeApYC1yam/dpsjP57gSOyE1fCOyenk8gC8hlwM+B7Zrc3+8DJ5VM2x1YmOvPTemxlGw4sJX780fALcDNZL9Mu5X2Mb2eTnbm4d2t7GN6n1YCN6bH3NL+tWsfltsnwByy0AbYPv2MLUs/cxNauN8OIhvuvjm376YDJ/X9PJLd4mpp2m/XAP+nxT97Zd+3kj6K7NO97k4/p5Nb2cfUh5Fkwfb83LS27UeyQF4DPJ3+Fp5Iduz5v4G7gN8BO6e2k4Hv5JZ9X/p5XAac0Op92a6Hb7FmZmY9rZeGRs3MzJ7DQWhmZj3NQWhmZj3NQWhmZj3NQWhmZj3NQWjWYpIukfSIpHI3JzCzFnMQmrXeV4D3trsTZpZxEJo1iaRXp5uVb5/ukrJU0ssj4r+B9e3un5llOvJeo2adICIWS1oAfAEYDpwXES27PZ6Z1cZBaNZcc8juO/ok8JE298XMyvDQqFlzjQZ2IPsU+O3b3BczK8NBaNZc3wI+S/Y5iV9qc1/MrAwPjZo1iaR/BJ6OiJ9IGgL8UdIbgX8DXgrsIGkVcGJE9MbH3ZgVkD99wszMepqHRs3MrKc5CM3MrKc5CM3MrKc5CM3MrKc5CM3MrKc5CM3MrKc5CM3MrKf9f8sw6qgDrrMSAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 1152x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Plotting the decision boundary\n",
    "\n",
    "plotModel(model.predict, 'fcnn separation with hidden layer keras') \n",
    "plt.scatter(X[0][0],X[0][1],c=\"black\",s=50)\n",
    "#plt.vlines(X[0][0],-10,10)\n",
    "#plt.hlines(X[0][1],-10,10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}