Deep_Learning_intro_part1_linear_perceptron.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction to Deep Learning\n",
    "## Part 1 - Linear perceptron and the concept of learning\n",
    "\n",
    "Discussion is on the structure of a neuron and the algorithmic concept concept behind learning.\n",
    "The coded example single linear neuron or perceptron, learning the logic OR and AND Gates."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import utils_plot as uplot\n",
    "import matplotlib.gridspec as gridspec\n",
    "from sklearn.metrics import accuracy_score, log_loss\n",
    "%matplotlib inline  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Inspiration - The Biological Neuron Model\n",
    "**The Neuron cell takes a binary input (of electrical signals), process it, and produce binary output (of electrical signal).**<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![title](https://raw.githubusercontent.com/naomifridman/Introduction_to_deep_learning/master/imgs/neuron3.PNG)\n",
    "\n",
    "**Neuron Input**<br>\n",
    "A neuron accepts inputs, usually from other neurons, through its dendrites, that are connect to other neurons via a gap called a synapse which assigns a \"weight\" to any specific input. <br>\n",
    "All of these inputs are processed together in the cell body (soma)<br>\n",
    "\n",
    "**Neuron Output**<br>\n",
    "The Neurons process the combination of the inputs, and if it exceeds a certain threshold, an output signal is produced (the neuron \"fires\")<br>\n",
    "When the neuron fires, the signal (output) travels along the axon to the synapse terminals,there it assigned a \"weight\", and continue to the dendrites of other neurons through the synapse.<br>\n",
    "With this biological model in mind, we can define a mathematical artificial neuron."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Perceptron - An Artificial Neural Network\n",
    "As in the Biological model, we want the neuron to intake an inputs, process it (weighted sum) and if it exceeds a threshold, the neuron will produce an output. \n",
    "\n",
    "![title](https://raw.githubusercontent.com/naomifridman/Introduction_to_deep_learning/master/imgs/perceptron_paradigma3.png)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Decision Units - Activation functions\n",
    "There are many options for decision units, and we will see later. Lets start with the basic one, defined by McCulloch-Pitts (1943)\n",
    "### Binary threshold unit\n",
    "![title](https://raw.githubusercontent.com/naomifridman/Introduction_to_deep_learning/master/imgs/part1_form1.JPG)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbYAAAC6CAYAAADCvGi/AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAMTQAADE0B0s6tTgAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAHrhJREFUeJzt3X9UVGX+B/D3KIiBAm1KCsMwNjr+qHH4kUSaSgmls2ruogQLKUpiaVuG2bqtftXWXHcrtrbUOHsQPLpx2MCNIn+Uq6KWrpgStijRCsIYu5SWAfJD4vn+wXKXYWYAkWHmDu/XOXPgzvPcez/zNId3z73XexVCCAEiIiInMcDeBRAREfUmBhsRETkVBhsRETkVBhsRETkVBhsRETkVBhsRETkVBhsRETkVBhtROwkJCYiPj7d3Gd1y5MgRKBQKNDc39+l+FQoFDh482OP1MzIyoFQqO+2jVCqRkZHR431Q/8Zgo34jPDwcCoUCCoUCAwcOhFKpxLPPPovGxkapzxtvvIGtW7fasUrL1q5di/DwcHuXQSQLDDbqV1auXImqqipUVFQgIyMDe/bswW9/+1up3cvLC15eXjbb/40bN2DPm/20D3EiZ8Vgo37Fw8MDI0aMgJ+fHyIiIhAVFYWzZ89K7R0PRarVarz22mtYsGABPDw8MH78eBw6dEhqv3DhAgwGA4YNGwZvb28YDAaUlZVJ7W2HC/fv34+7774bt912G/7yl7/A19cXP/74o9RPCIGAgACLh98yMjLw8ssvIz8/X5pxlpeXS+35+fmYMGEChg4dinnz5uG7776T2sLDw/H8889j6dKl8PT0xKpVqwAAFy9exJw5czBkyBD4+vri6aefxvXr16X1Xn/9dYwaNQpubm5QKpXYsGGDSU1GoxERERFwd3dHSEgIioqKTNpfeeUV+Pv7w83NDWFhYTh16pTV/yZNTU1ISkrCkCFD4O/vj127dlntS9QdDDbqtyorK3Hw4EFMmjSp035/+MMfMGfOHBQWFmLq1KmIj49HU1MTAKC2thbz58/H8ePHcfz4cQwaNAgxMTFm29i4cSP+/Oc/49y5c5g7dy7q6+tNzlPl5+fj22+/RVRUlNm6jz32GFauXIn7778fVVVVqKqqgr+/v9S+adMmZGRk4PDhwzh37hw2bdpksn5qaio0Gg3OnDmDVatWoampCY888gjGjBmDzz77DLm5uSgoKJBCr6CgAOvXr8fbb7+N0tJS/PWvf8Xo0aNNtvnSSy/hl7/8JQoLC+Hr64vFixdLbe+88w42bNiALVu2oLCwEBMnToTBYMAPP/xgcXx/97vf4YMPPsCePXuQl5eHtLQ0XLlypdP/JkSdEkT9xPTp04Wrq6vw8PAQgwcPFgBEZGSkaGpqkvosWrRIxMXFScsBAQHiqaeekpa//vprAUCcO3fO4j6qqqoEAHHp0iUhhBCHDx8WAMSRI0dM+i1btkz84he/kJYXL15sst+OfvOb34jp06ebvNe27X/84x/Se5s3bxYhISEmnzk8PNxkvZ07d5r0EUKITz75RAwaNEg0NzeL7OxsodVqxY0bNyzWAkD8/ve/l5Y//fRTAUDU1NQIIYS47777xOrVq6X2GzduCKVSKd566y0hhBDp6enCz89Pavfx8RHbt2+Xls+fPy8AiPT0dKvjQdQZztioX1m6dCkKCwvx+eefY//+/TAajVi5cmWn6+h0Oun3ESNGAACqq6sBANeuXcOKFSswZswYeHp6SjObyspKk20EBQWZLCckJOC9995DTU0N6uvrkZOTg4ULF/boM3Wsr602a/s+d+4cPv/8cwwZMkR6RUZGoqmpCZcvX0ZERAQUCgU0Gg2efPJJfPjhh2bnBTsbk5KSEoSFhUntLi4uuPfee1FSUmJW+7Vr11BdXY3Q0FDpvXHjxmHo0KE3OwxEEhd7F0DUl26//XYpfLRaLdavX4+4uDi8+uqruO222yyu4+rqKv2uUCgAAC0tLQCAVatW4eTJk9I5qebmZuj1ety4ccNkG+7u7ibLYWFhUKlUyMnJgZubG4YOHYqIiIgefaaO9bXVZm3ftbW1mDZtGlJTU822NXLkSLi6uqKoqAgHDx7E/v37sWTJEtx33314//33re4TgNl+u6MtMNu2QdQbGGzUr7m4uODHH39EU1OT1WDrzMmTJ/HEE0/gpz/9KQDg2LFj3V43ISEBu3btgpubG+Li4jBggPUDKK6uriYXm9wKvV6P999/H0qlEoMHD7bYZ9CgQTAYDDAYDIiPj8d9992H6upq+Pj4dLn9sWPH4uTJk/j5z38OAGhubsbp06cRGRlp1tfb2xs+Pj44deqUNLMsKSlBTU3NLXxC6u94KJL6lbq6Ovz73/9GVVUVPvnkE2zatAkPPPBAjy/x12g0yM7ORnFxMY4fP47Vq1d3e93HH38cx44dw0cffdTlYciAgACUlJTgwoUL+Pbbb3s0O2oTFxeHQYMG4bHHHkNBQQG++uorfPDBB3j++ecBAHl5edi6dSvOnTuHixcvIisrC8OGDcMdd9zRre0/++yz2LZtG9555x1cuHABy5cvR319vdV/+P7kk09i48aNOHjwID7//HM89dRTVgOXqDsYbNSvvP766xg5ciT8/Pwwf/583H333cjKyurx9l577TUIIRASEoKkpCS89NJL3V7X19cXDz30EPR6Pe6+++5O+86fPx+hoaGYNGkShg8fjoqKih7XPHToUBw5cgSDBg1CZGQk9Ho91q5di5EjRwJonUVlZWVh6tSpmDhxIk6dOoW8vDwMHDiwW9uPjY3F+vXr8cILL0Cv16OoqAh79+6Fp6enxf4vvvgiZs2ahUcffRQGgwELFy7sdogSWaIQHc8KE1Gf0ev1SExMxDPPPGPvUoicBs+xEdnB1atXkZ2dja+++qrHV0MSkWUMNiI7CA4Oxg8//IBt27bB29vb3uUQORUeiiQiIqfCi0eIiMipMNiIiMipOPU5Njc3NwwfPtzeZRAR0S365ptvuv3YJacOtuHDh8NoNNq7DCIiukVdPXW9PZsfinzmmWegVquhUCjwxRdfWO23adMmaDQaaDQarFu3rtttRERE7dk82NqeVRUQEGC1z9GjR5GZmYmioiIUFxdj3759OHDgQJdtREREHdn8UOS0adO67JOVlYWEhAR4eHgAAJYsWYLMzEw88sgjnbYREdDSAuTlAdeu2bsSIsuGDwdmzuy7/TnEObaKigpMnz5dWlar1cjOzu6yraOUlBSkpKRIy7W1tTaqmMhxHD8OPPqovasgsm7y5H4YbIDp85g6/pvxztraS05ORnJysrR8MycbieTq6tXWn6tWAe3+H5DIYdx+e9/uzyGCTaVSoby8XFq+dOkSVCpVl21EBDQ0tP68/35gzhz71kLkCBziH2gvWLAAO3fuRF1dHRobG7Fjxw7ExMR02UZE/ws2PsKMqJXNg23FihVQKpUwGo2IiIjA6NGjAQAGgwGnT58GAISHhyM6Oho6nQ7jx4/Hww8/jJn/PSDbWRsRMdiIOnLqmyC3BSqRM3v9deC551ovIpkyxd7VENnGzfw9d4hDkUTUc5yxEZlisBHJHIONyBSDjUjmGGxEphhsRDLHYCMyxWAjkjkGG5EpBhuRzDHYiEwx2Ihkri3Y3NzsWweRo2CwEclcYyPg4tL6IiIGG5HsNTTwMCRReww2IpljsBGZYrARyRyDjcgUg41I5hoaeOEIUXsMNiKZ44yNyBSDjUjmGGxEphhsRDLHYCMyxWAjkjkGG5EpBhuRzDHYiEwx2IhkTAgGG1FHDDYiGWtqav3JYCP6H5sHW2lpKSZPngytVovQ0FAUFxeb9dmyZQsCAwOll6enJ5KTkwEAR44cgbu7u0l7fX29rcsmkgXe2Z/InM1vm7ps2TIkJSUhISEB2dnZSExMxIkTJ0z6rFmzBmvWrAEANDU1wdfXF3FxcVL7hAkTcPr0aVuXSiQ7DDYiczadsVVXV+PMmTOIj48HAERFRaGsrAzl5eVW13nvvfegVCoREhJiy9KInAKDjcicTYOtsrISvr6+cPnv8zQUCgVUKhUqKiqsrpOWlobExEST90pKShAcHIxJkyZh27ZtVtdNSUmBUqmUXrW1tb3zQYgcFIONyJzND0UqFAqTZSGE1b6VlZU4fvw4MjMzpfeCg4NhNBrh5eUFo9EIg8GAYcOGITo62mz95ORk6dwcACiVyl74BESOi8FGZM6mMzZ/f38YjUY0NzcDaA21yspKqFQqi/3T09Mxd+5c/OQnP5He8/T0hJeXF4DWoIqNjcWxY8dsWTaRbDDYiMzZNNh8fHwQFBSE3bt3AwBycnKgVquhVqvN+gohkJGRYXYYsqqqCi0tLQCAmpoa5OXlISgoyJZlE8kGg43InM0v909NTUVqaiq0Wi22bNmCtLQ0AIDBYDC50vHQoUMQQmDGjBkm6+fk5ECn00Gv1yMsLAyRkZFYvHixrcsmkgUGG5E5hejspJfMKZVKGI1Ge5dBZDO5ucC8ecCuXcB/Lz4mcko38/ecdx4hkjHO2IjMMdiIZIzBRmSOwUYkYww2InMMNiIZY7ARmWOwEckYg43IHIONSMYYbETmGGxEMtbY2PqTwUb0Pww2IhnjjI3IHIONSMYYbETmGGxEMtYWbG5u9q2DyJEw2IhkjDM2InMMNiIZa2gABgwAXGz+ZEUi+WCwEclYQ0PrbK3D83yJ+jUGG5GMtQUbEf0Pg41IxhhsROYYbEQyxmAjMsdgI5IxBhuROQYbkYwx2IjM2TzYSktLMXnyZGi1WoSGhqK4uNisT0ZGBry9vREYGIjAwEA8+OCDJu2bNm2CRqOBRqPBunXrbF0ykWww2IjMdRpsf/zjH295B8uWLUNSUhK+/PJLvPDCC0hMTLTYLyIiAoWFhSgsLMThw4el948ePYrMzEwUFRWhuLgY+/btw4EDB265LiJnwGAjMtdpsB04cADh4eGoqKjo0carq6tx5swZxMfHAwCioqJQVlaG8vLybm8jKysLCQkJ8PDwgJubG5YsWYLMzMwe1UPkbBhsROY6Dbb9+/cjLi4OU6ZMQXp6+k1vvLKyEr6+vnD5720RFAoFVCqVxaDMz89HYGAgpkyZguzsbOn9iooKBAQESMtqtdpq0KakpECpVEqv2tram66ZSC6EYLARWdLljXiWLl2K8PBwhIaGYvXq1RgwYACEEFAoFKiuru5yB4oOt0QQQpj1mT17NqKjo+Hu7o7z58/j4YcfhlKpRFhYmNk2LK3fJjk5GcnJydKyUqnssj4iubpxozXcGGxEproMts8++wwJCQmIiYnB6tWrMXDgwG5v3N/fH0ajEc3NzXBxcYEQApWVlVCpVCb9hg0bJv0+fvx4GAwGfPLJJwgLC4NKpTI5dHnp0iWz9Yn6I94AmciyTg9Frl27FvPnz8err76K7du346677kJAQID06oqPjw+CgoKwe/duAEBOTg7UajXUarVJv8uXL0u//+c//8GhQ4cQFBQEAFiwYAF27tyJuro6NDY2YseOHYiJibnZz0nkdBhsRJZ1OmMrLy/H2bNn4e3t3eMdpKamIiEhAZs3b4anpyd27twJADAYDHjppZdw7733YuvWrcjNzYWrqytaWlrw3HPP4aGHHgIAhIeHIzo6GjqdDgAQExODmTNn9rgeImfBYCOyTCE6O2klc0qlEkaj0d5lENnEl18CY8cCL74IvPyyvashsq2b+XvOO48QyRRnbESWMdiIZIrBRmQZg41IphhsRJYx2IhkisFGZBmDjUimGGxEljHYiGSKwUZkGYONSKYaG1t/MtiITDHYiGSKMzYiyxhsRDLFYCOyjMFGJFNtwebmZt86iBwNg41IpjhjI7KMwUYkUww2IssYbEQyxWAjsozBRiRTDDYiyxhsRDLFYCOyjMFGJFMMNiLLGGxEMtXQACgUgKurvSshciwMNiKZamhona0pFPauhMixMNiIZKot2IjIlM2DrbS0FJMnT4ZWq0VoaCiKi4vN+mRlZSEoKAj33HMPdDod3nzzTantyJEjcHd3R2BgoPSqr6+3ddlEDo/BRmSZi613sGzZMiQlJSEhIQHZ2dlITEzEiRMnTPoolUrs27cPI0aMwLVr1xASEoLg4GBMmTIFADBhwgScPn3a1qUSyQqDjcgym87YqqurcebMGcTHxwMAoqKiUFZWhvLycpN+U6ZMwYgRIwAAXl5eGDduHMrKymxZGpHsMdiILLNpsFVWVsLX1xcuLq0TQ4VCAZVKhYqKCqvrFBcX48SJE3jooYek90pKShAcHIxJkyZh27ZtVtdNSUmBUqmUXrW1tb33YYgcDIONyDKbH4pUdLhkSwhhta/RaMSjjz6Kt99+G76+vgCA4OBgGI1GeHl5wWg0wmAwYNiwYYiOjjZbPzk5GcnJydKyUqnspU9B5HgaGoA777R3FUSOx6YzNn9/fxiNRjQ3NwNoDbXKykqoVCqzvl9//TUiIiKwdu1aLFiwQHrf09MTXl5eAFqDKjY2FseOHbNl2USywBkbkWU2DTYfHx8EBQVh9+7dAICcnByo1Wqo1WqTflVVVZgxYwZ+9atfYdGiRWZtLS0tAICamhrk5eUhKCjIlmUTyQKDjcgym1/un5qaitTUVGi1WmzZsgVpaWkAAIPBIF3p+H//93+oqKjAG2+8IV3Sn56eDqA1DHU6HfR6PcLCwhAZGYnFixfbumwihyYEg43IGoXo7KSXzCmVShiNRnuXQdTrbtwABg0CYmOBd96xdzVEtnczf8955xEiGeINkImsY7ARyRCDjcg6BhuRDDHYiKxjsBHJEIONyDoGG5EMNTa2/mSwEZljsBHJEGdsRNYx2IhkiMFGZB2DjUiG2oLNzc2+dRA5IgYbkQxxxkZkHYONSIYYbETWMdiIZIjBRmQdg41IhhhsRNYx2IhkiMFGZB2DjUiGGGxE1jHYiGSIwUZkHYONSIYYbETWMdiIZIjBRmQdg41IhhhsRNbZPNhKS0sxefJkaLVahIaGori42GK/TZs2QaPRQKPRYN26dd1uI+qPGGxE1tk82JYtW4akpCR8+eWXeOGFF5CYmGjW5+jRo8jMzERRURGKi4uxb98+HDhwoMs2ov6KwUZknYstN15dXY0zZ87go48+AgBERUXh6aefRnl5OdRqtdQvKysLCQkJ8PDwAAAsWbIEmZmZeOSRRzpts6W6OmD9epvugqjHCgpafw4aZN86iByRTYOtsrISvr6+cHFp3Y1CoYBKpUJFRYVJsFVUVGD69OnSslqtRnZ2dpdtHaWkpCAlJUVarq2t7XHt9fXAa6/1eHUim9NoAIXC3lUQOR6bBhvQGmbtCSG67NexT2dt7SUnJyM5OVlaViqVN1Vre7ffDpSX93h1Ipvz8bF3BUSOyabB5u/vD6PRiObmZri4uEAIgcrKSqhUKpN+KpUK5e1S5NKlS1KfztpsaeBAICDA5rshIqJeZtOLR3x8fBAUFITdu3cDAHJycqBWq00OQwLAggULsHPnTtTV1aGxsRE7duxATExMl21EREQd2fyqyNTUVKSmpkKr1WLLli1IS0sDABgMBpw+fRoAEB4ejujoaOh0OowfPx4PP/wwZs6c2WUbERFRRwrR2UkrmVMqlTAajfYug4iIbtHN/D136mBzc3PD8OHDb2kbtbW1GDJkSC9VZDuss/fJpVbW2bvkUicgn1p7o85vvvkGjY2N3err1MHWG+Qy62OdvU8utbLO3iWXOgH51NrXdfJekURE5FQYbERE5FQGbtiwYYO9i3B0999/v71L6BbW2fvkUivr7F1yqROQT619WSfPsRERkVPhoUgiInIqDDYiInIq/T7YduzYAZ1OBxcXF7z11lsmbdevX0dsbCxGjx4NrVaLPXv2WN1OXl4exo0bh9GjRyMqKuqWnizQHfPnz0dgYKD0GjBgAN5//32LfRUKBSZOnCj1PXbsmE1ray8hIQFKpVLa9+rVq6327esxbO/FF1/E+PHjodfrERoaikOHDlnta4/x7I0H9tpaQ0MD5s2bB61Wi8DAQMycOdPkPq9tjhw5And3d5Pvb319fZ/WqlarMW7cOGn/WVlZFvvZ+yHH33//vck4abVauLi44OrVqyb97DGmzzzzDNRqNRQKBb744gvp/e5+VwEbjq/o5woLC0VxcbF4/PHHxZtvvmnStnHjRrFo0SIhhBAXL14Ud955p7h69arZNmpqaoSPj484f/68EEKIFStWiDVr1ti89jYFBQXijjvuEA0NDRbbAYiampo+q6e9RYsWmY2rJfYew71794rr168LIVq/E97e3qK+vt5iX3uM54MPPijS09OFEEK8++67IiwszKxPfn6+mDBhgqitrRUNDQ0iJCRE7N+/v89qrK+vFx9++KFoaWkRQgjx5ptvisjISLN+hw8fFiEhIX1WlyUBAQHi3Llznfax93ha8sorr4jZs2ebvW+PMc3PzxeVlZVmY9md72rb+rYa334/Y9Pr9Rg/fjwGDDAfiqysLKxYsQIAMGrUKEybNg25ublm/fbt24d7770X48aNAwAsX74cmZmZti28nR07diA+Ph5ubm59ts/eZu8xnDVrFm677TYAgE6nw48//ohvv/22z/bfmbYH9sbHxwNofWBvWVmZ2Wyo/UN53dzcpIfy9pXBgwfDYDBIj5kKCwvDxYsX+2z/vc3e42lJeno6EhMT7VpDm2nTppk9Gqy731XAtuPb74OtMxUVFQho9+watVqNioqKbvW7fPkyWlpabF5jQ0MDMjMzu/yyh4eHQ6/XIzk5GXV1dTavq72UlBRMnDgRs2fPRmFhocU+9hzDjtLT06HRaDp9nl9fjmdnD+xtr7vf177ypz/9CXPmzLHYVlJSguDgYEyaNAnbtm3r48paxcXFQafT4YknnsA333xj1u5o43nixAlcuXIFs2fPttjuCGPa3e8qYNvxtfmDRu1t6tSpOH/+vMW2s2fPwt/fv9P1u/uQ044PVL1V3a07JycHY8aMgU6ns7qttmfY1dXV4cknn8Tq1at77YvfVZ0vv/wyRo4ciQEDBuBvf/sbZs2ahdLSUov3jevtMbyZOtvG8+9//zs2btyIjz/+2Oq2bDme1vTGA3v70ubNm1FaWoq3337brC04OBhGoxFeXl4wGo0wGAwYNmwYoqOj+6y+o0ePQqVS4caNG1i7di0WLVqEvXv3mvVzlPEEWo/MLFy4UAqN9hxhTNt097vasW9vjq/TB9utnNhve8hp242UL126BIPBYLFf+4sNysvL4efnZ/HwZnd1t+60tLQuZ2ttD2b18PDA8uXLkZSU1OO6OrqZ8f3Zz36GNWvWoKSkBCEhIWY19vYY3myd+fn5WLx4MT744AOMHTvWaj9bjqclvfHA3r706quvYs+ePTh48CDc3d3N2j09PaXflUolYmNjcezYsT79I9w2Lq6urli5ciW0Wq3FPo4wngBQV1eHrKwsnDp1ymK7I4wp0P3vKmDb8eWhyE4sWLAAW7duBQCUlZUhPz8fc+fONes3c+ZMFBQU4MKFCwCAbdu29cnDUMvKynDq1CnExsZa7fPdd9/h+vXrAICWlhZkZWUhKCjI5rW1aX/j05MnT+LKlSsYPXq0WT97jWGbo0eP4vHHH0dubi70er3VfvYYz954YG9fSUlJQWZmJj7++GN4e3tb7FNVVSUdYq6pqUFeXl6ffifr6urw/fffS8uZmZkW9+8I49nm3XffxcSJE6Vz0B3Ze0zbdPe7Cth4fHvlEhQZ27Vrl/Dz8xPu7u7C29tb+Pn5iTNnzgghhKitrRXR0dFCo9GIMWPGiHfffVdab926dWL79u3Scm5urhg7dqzQaDRi3rx54tq1azavfe3atWLhwoVm72/fvl2sW7dOCCHEp59+KnQ6nZg4caKYMGGCiI+PF1euXLF5bW1mzJgh7rnnHqHX60VYWJg4dOiQ1OYIY9hm9OjRwsfHR+j1eulVVFQkhHCM8bxw4YIICwsTY8aMESEhIeKLL74QQggxa9YsUVBQIPXbuHGjGDVqlBg1apT49a9/bfO62qusrBQAxF133SWNYWhoqBBCiMTERJGbmyuEaL1acsKECdIYrl+/XrqSsi/861//EoGBgUKn04l77rlHzJ07V5SVlQkhHGs823vggQfEjh07TN6z95guX75c+Pn5iYEDB4o777xTaDQaIYT176oQfTe+vKUWERE5FR6KJCIip8JgIyIip8JgIyIip8JgIyIip8JgIyIip8JgIyIip8JgI5KJ77//HiqVCidOnJDee+utt/Dggw/a/XZPRI6E/46NSEb27t2L5557DmfPnsXly5cxdepUnDx50uKdHYj6KwYbkcwkJiZi8ODBOHv2LBYvXoylS5fauyQih8JgI5KZH374AWq1GiEhIZ0+hYCov+I5NiKZyc/Ph7u7Oy5evIja2lp7l0PkcDhjI5KRK1euIDAwEDk5Odi5cydaWlqwfft2e5dF5FAYbEQyEhMTg7vuugubN29GXV0d9Ho9UlNTMWPGDHuXRuQweCiSSCays7Pxz3/+Exs2bADQ+qDTtLQ0LF26FDU1NfYtjsiBcMZGREROhTM2IiJyKgw2IiJyKgw2IiJyKgw2IiJyKgw2IiJyKgw2IiJyKgw2IiJyKgw2IiJyKv8PmTQhjefWoMEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 480x160 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "uplot.drow_binary_threshold()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Learning the Bias\n",
    "Now we need to learn the weights and the bias. But we can use a trick, to avoid separate scheme, for learning the bias. \n",
    "![title](https://raw.githubusercontent.com/naomifridman/Introduction_to_deep_learning/master/imgs/part1_form2.JPG)\n",
    "> This way, we can learn the bias as it was part of the weights. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Learning weights\n",
    "The idea is to go over the samples, and correct/update the weights according to the results on the samples.\n",
    "![title](https://raw.githubusercontent.com/naomifridman/Introduction_to_deep_learning/master/imgs/dl3.jpg)\n",
    "#### Update the weights\n",
    "There are many strategies to update the weights, and we will see them later."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "### Perceptron Example - Logic OR\n",
    "Lets build a Perceptron, that performs a logic OR:\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "X1 | X2 | OR\n",
    "--- | --- | ---\n",
    "*0* | 0 | **0**\n",
    "*1* | 0 | **1**\n",
    "*0* | 1 | **1**\n",
    "*1* | 1 | **1**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* $X$ is the input\n",
    "* $w$ are the weights\n",
    "* $X*w$ is the input to the activation function\n",
    "$$z = Xw$$\n",
    "* $\\hat{y}$ is the 0/1 output from activation\n",
    "$$ \\hat{y} = \\begin{cases} 1 & z \\geqslant  0\\\\ 0 & otherwise \\end{cases}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "The scheme will be to Go over all training cases in any order, and:\n",
    "  * If the output is correct, continue\n",
    "  * If output < label  (increase $w$): $\\to{w} = {w} + {X}_{i}$\n",
    "  * If output > label (decrease $w$): $\\to{w} = {w} - {X}_{i}$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The activation function\n",
    "def binary_threshold_activation(Xw):\n",
    "    yhat = 0\n",
    "    if(Xw > 0):\n",
    "        yhat = 1\n",
    "    return yhat"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "def predict(X, w):\n",
    "    yhat = 0\n",
    "    z = np.dot(X, w)\n",
    "    if(z > 0):\n",
    "        yhat = 1\n",
    "    return yhat"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "def perceptron_train(X, Y,n=7):\n",
    "\n",
    "    w = np.zeros(len(X[0]))\n",
    "    errors = []\n",
    "    \n",
    "    for t in range(n):\n",
    "        \n",
    "        print('--------------------------\\nepoch: ' , t)\n",
    "        total_error = 0\n",
    "        \n",
    "        for i in range(X.shape[0]):\n",
    "             \n",
    "            z = np.dot(X[i], w)\n",
    "            yhat = binary_threshold_activation(z)\n",
    "            \n",
    "            if (yhat == Y[i]):\n",
    "                print(i,'x:',X[i],'w:', w,'Wx: ', np.dot(X[i], w),\n",
    "                      'y:',Y[i], end=' ')\n",
    "                print('correct no update')\n",
    "                continue\n",
    "                \n",
    "            elif (yhat < Y[i]):\n",
    "                total_error += 1\n",
    "                print(i, 'yhat<y', 'X:',X[i],'w:', w,'Wx:',\n",
    "                      np.dot(X[i], w), 'y:',Y[i], end=' ')\n",
    "        \n",
    "                \n",
    "                w = w + X[i]\n",
    "                print('=> W updated:', w)\n",
    "                \n",
    "            elif (yhat > Y[i]):\n",
    "                total_error += 1\n",
    "                print(i, 'yhat>y' ,'X:',X[i],'w:', w,'Wx:', \n",
    "                      np.dot(X[i], w), 'y:',Y[i] ,end = ' ')\n",
    "                \n",
    "                w = w - X[i]\n",
    "                print('=> W updated:', w)\n",
    "                \n",
    "        errors.append(total_error)\n",
    "        \n",
    "    plt.plot(errors)\n",
    "    plt.xlabel('Epoch')\n",
    "    plt.ylabel('Total Loss')\n",
    "    \n",
    "    return w"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--------------------------\n",
      "epoch:  0\n",
      "0 x: [1 0 0] w: [0. 0. 0.] Wx:  0.0 y: 0 correct no update\n",
      "1 yhat<y X: [1 1 0] w: [0. 0. 0.] Wx: 0.0 y: 1 => W updated: [1. 1. 0.]\n",
      "2 x: [1 0 1] w: [1. 1. 0.] Wx:  1.0 y: 1 correct no update\n",
      "3 x: [1 1 1] w: [1. 1. 0.] Wx:  2.0 y: 1 correct no update\n",
      "--------------------------\n",
      "epoch:  1\n",
      "0 yhat>y X: [1 0 0] w: [1. 1. 0.] Wx: 1.0 y: 0 => W updated: [0. 1. 0.]\n",
      "1 x: [1 1 0] w: [0. 1. 0.] Wx:  1.0 y: 1 correct no update\n",
      "2 yhat<y X: [1 0 1] w: [0. 1. 0.] Wx: 0.0 y: 1 => W updated: [1. 1. 1.]\n",
      "3 x: [1 1 1] w: [1. 1. 1.] Wx:  3.0 y: 1 correct no update\n",
      "--------------------------\n",
      "epoch:  2\n",
      "0 yhat>y X: [1 0 0] w: [1. 1. 1.] Wx: 1.0 y: 0 => W updated: [0. 1. 1.]\n",
      "1 x: [1 1 0] w: [0. 1. 1.] Wx:  1.0 y: 1 correct no update\n",
      "2 x: [1 0 1] w: [0. 1. 1.] Wx:  1.0 y: 1 correct no update\n",
      "3 x: [1 1 1] w: [0. 1. 1.] Wx:  2.0 y: 1 correct no update\n",
      "--------------------------\n",
      "epoch:  3\n",
      "0 x: [1 0 0] w: [0. 1. 1.] Wx:  0.0 y: 0 correct no update\n",
      "1 x: [1 1 0] w: [0. 1. 1.] Wx:  1.0 y: 1 correct no update\n",
      "2 x: [1 0 1] w: [0. 1. 1.] Wx:  1.0 y: 1 correct no update\n",
      "3 x: [1 1 1] w: [0. 1. 1.] Wx:  2.0 y: 1 correct no update\n",
      "--------------------------\n",
      "epoch:  4\n",
      "0 x: [1 0 0] w: [0. 1. 1.] Wx:  0.0 y: 0 correct no update\n",
      "1 x: [1 1 0] w: [0. 1. 1.] Wx:  1.0 y: 1 correct no update\n",
      "2 x: [1 0 1] w: [0. 1. 1.] Wx:  1.0 y: 1 correct no update\n",
      "3 x: [1 1 1] w: [0. 1. 1.] Wx:  2.0 y: 1 correct no update\n",
      "--------------------------\n",
      "epoch:  5\n",
      "0 x: [1 0 0] w: [0. 1. 1.] Wx:  0.0 y: 0 correct no update\n",
      "1 x: [1 1 0] w: [0. 1. 1.] Wx:  1.0 y: 1 correct no update\n",
      "2 x: [1 0 1] w: [0. 1. 1.] Wx:  1.0 y: 1 correct no update\n",
      "3 x: [1 1 1] w: [0. 1. 1.] Wx:  2.0 y: 1 correct no update\n",
      "--------------------------\n",
      "epoch:  6\n",
      "0 x: [1 0 0] w: [0. 1. 1.] Wx:  0.0 y: 0 correct no update\n",
      "1 x: [1 1 0] w: [0. 1. 1.] Wx:  1.0 y: 1 correct no update\n",
      "2 x: [1 0 1] w: [0. 1. 1.] Wx:  1.0 y: 1 correct no update\n",
      "3 x: [1 1 1] w: [0. 1. 1.] Wx:  2.0 y: 1 correct no update\n",
      "Learned Weights: [0. 1. 1.]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYwAAAEKCAYAAAAB0GKPAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xl8V/Wd7/HXJxuBsJOwB8KSULEq2ggqimyhOF102mnVjlarU0TFojhLZ+6903vtncedmY6gVmuLStXWam1rW9vakV1QRAgoKlKSELYAQth3yPK5f+TQpjQJP5Lfycnvl/fz8TiP/M73bO8fCp98z/Y1d0dERORcUqIOICIiiUEFQ0REYqKCISIiMVHBEBGRmKhgiIhITFQwREQkJioYIiISExUMERGJiQqGiIjEJC3qAPGUnZ3teXl5UccQEUkYa9as2evuObGsm1QFIy8vj+Li4qhjiIgkDDPbGuu6OiUlIiIxUcEQEZGYqGCIiEhMVDBERCQmKhgiIhKT0AqGmeWa2RIz22Bm681sZgPrmJk9ZmZlZva+mV1Wb9ltZlYaTLeFlVNERGIT5m211cCD7r7WzLoAa8xsgbt/VG+d64D8YBoDPAmMMbOewLeAQsCDbV919wMh5hURkSaE1sNw913uvjb4fATYAAw4a7Xrgee9zkqgu5n1Az4NLHD3/UGRWABMDStrsjhVXcOPVm7lxOmaqKOISBJqlWsYZpYHXAq8c9aiAcD2evMVQVtj7Q3te5qZFZtZcWVlZbwiJ6Rn39rC//rVh8xdVh51FBFJQqEXDDPrDPwCuN/dD5+9uIFNvIn2v2x0n+vuhe5emJMT09PtSenoqWq+/8YmAJ5eXs7B46cjTiQiySbUgmFm6dQVixfc/ZUGVqkAcuvNDwR2NtEujfjhm5s5cLyK//zixRw9Xc1Ty9XLEJH4CvMuKQOeATa4++xGVnsV+Gpwt9QVwCF33wW8Dkwxsx5m1gOYErRJAw4dr2Lu8nImX9CHL1+ey2cu6scP39rCvqOnoo4mIkkkzB7GWOBWYKKZvRdMf2Vm081serDOa0A5UAY8BdwD4O77gW8Dq4PpoaBNGvD0m+UcOVnNrKICAO6fXMDJqpo/nqISEYmH0G6rdfc3afhaRP11HLi3kWXzgHkhREsq+4+dZt6bm/nMRf0Y2b8rAMN7d+aGSwfw/Ntb+fo1Q+ndNTPilCKSDPSkd4L7wRubOF5Vw/2T8/+sfeakfKprnSeWlEWUTESSjQpGAttz5CTPvb2FG0YNIL9Plz9bNrhXFl8uHMiLq7az4+CJaAKKSFJRwUhgTy7dRFWNM3NSfoPLZ0ysa398sXoZItJyKhgJatehE7ywcht/c9lA8rKzGlxnQPeO3Dw6l58Vb2fbvuOtnFBEko0KRoJ6fHEZjnPfpOFNrnfvhOGkphiPLiptpWQikqxUMBLQ9v3H+enq7dx4eS4De3Rqct3eXTO59YrB/PLdCsr2HG2lhCKSjFQwEtBji0pJSTFmTGj42sXZpo8fRmZ6qnoZItIiKhgJprzyKK+8u4Nbxgymb7fYnq/I7tyB26/K47fv7+QPH5/9Oi8RkdioYCSYRxeVkpGawt3jh53XdtPGDaVzRhpzFpSElExEkp0KRgIp2X2EV9ft5Lar8sjp0uG8tu3eKYM7rxnC6+t380HFoZASikgyU8FIII8sLCErI427xg1t1vZ3XD2Ebh3Tmb1gY5yTiUh7oIKRINbvPMRrH3zMHVcPoUdWRrP20TUznbuuHcqSjZWs2arRbkXk/KhgJIg5C0rompnGnVcPadF+brsyj15ZGbqWISLnTQUjAby77QALN+xh2rihdOuY3qJ9ZXVI4+7xw3izbC8ry/fFKaGItAcqGAlg9oISemZlcPvYlvUuzrjlisH06dqB2fNLqHvDvIjIuYU54t48M9tjZh82svwf6g2s9KGZ1ZhZz2DZFjP7IFhWHFbGRLBq836Wl+5l+rVD6dwhPsOXZKancu+E4azaUrdvEZFYhNnDeBaY2thCd/+Ou49y91HAPwNvnDWq3oRgeWGIGds0d+fh+RvJ6dKBW6/Ii+u+b7w8lwHdO/LwAvUyRCQ2oRUMd18GxDqs6s3Ai2FlSVQrNu3jnc37uXf8MDpmpMZ13x3SUrlv4nDWbT/I4j/sieu+RSQ5RX4Nw8w6UdcT+UW9Zgfmm9kaM5sWTbJoneld9OuWyU2jB4VyjC9+aiCDe3Xi4fkl1NaqlyEiTYu8YACfA94663TUWHe/DLgOuNfMxjW2sZlNM7NiMyuurKwMO2urWbqxkrXbDnLfxHwy0+PbuzgjPTWFmZPy+WjXYV5f/3EoxxCR5NEWCsZNnHU6yt13Bj/3AL8ERje2sbvPdfdCdy/MyckJNWhrcXceXrCR3J4d+VLhwFCPdf2oAQzLyWLOwhJq1MsQkSZEWjDMrBtwLfDrem1ZZtblzGdgCtDgnVbJ6vX1u/lwx2FmTiogPTXc/0SpKcYDRQWU7D7Kb9/fGeqxRCSxhXlb7YvA28AIM6swszvNbLqZTa+32l8D8939WL22PsCbZrYOWAX8zt3/O6ycbU1trTNnQQlDs7O4YVT/VjnmX32yH5/o24VHFpZSXVPbKscUkcQTnxv7G+DuN8ewzrPU3X5bv60cuCScVG3f7z7YxcbdR3j0plGkhdy7OCMlxZhVVMC0H63hlXd38OXC3FY5rogklrZwDUMC1TW1zFlYwog+Xfjcxa3TuzijaGQfLh7YjccWlXK6Wr0MEflLKhhtyK/f20l55TEeKMonJcVa9dhmddcyKg6c4GdrtrfqsUUkMahgtBFVNbU8uqiUC/t35dMX9o0kw/iCHD41uAffXVTGyaqaSDKISNulgtFG/HxNBdv2H+fBKQWYtW7v4gwz48GiAj4+fJIXV22LJIOItF0qGG3AqeoavruolFG53ZkwonekWa4ans2VQ3vxxJJNnDitXoaI/IkKRhvw0qrt7Dx0kr+fMiKy3kV9D04pYO/RUzz/9paoo4hIG6KCEbGTVTU8saSM0UN6MnZ4r6jjAFCY15NxBTl8/41NHD1VHXUcEWkjVDAi9uOVW9lz5BQPFkV37aIhDxYVcOB4FT98c3PUUUSkjVDBiNCxU9V8b+kmrsnPZszQttG7OOOS3O5MvqAPc5eXc+h4VdRxRKQNUMGI0LMrtrD/2GlmFRVEHaVBs4oKOHKymqffLI86ioi0ASoYETl8soq5y8qZ9IneXDqoR9RxGjSyf1c+c1E/5r25mf3HTkcdR0QipoIRkWeWb+bQiSoeaKO9izPun5zP8aoafrBsU9RRRCRiKhgROHDsNPPe3MzUC/vyyQHdoo7TpPw+Xbhh1ACeW7GFPUdORh1HRCKkghGBucvLOXq6us33Ls6YOSmfqhrnyaXqZYi0ZyoYrWzv0VM8+9YWPndxf0b07RJ1nJjkZWfxN5cN5IV3trHr0Imo44hIRFQwWtmTSzdxqrqG+yfnRx3lvNw3aTjuzuOLy6KOIiIRCXPEvXlmtsfMGhxe1czGm9khM3svmP613rKpZrbRzMrM7JthZWxtuw+f5Mcrt/KFywYyNKdz1HHOy8Aenbjx8lxeLt7O9v3Ho44jIhEIs4fxLDD1HOssd/dRwfQQgJmlAk8A1wEjgZvNbGSIOVvNE0vKqKl1Zk5KrN7FGTMm5GNmPLaoNOooIhKB0AqGuy8D9jdj09FAmbuXu/tp4CXg+riGi0DFgeO8uGobX748l9yenaKO0yx9u2Vyy5jBvPLuDjbvPXbuDUQkqUR9DeNKM1tnZr83swuDtgFA/SHfKoK2hPb44jIMY8aE4VFHaZG7xw8jIzWFRxeWRB1FRFpZlAVjLTDY3S8Bvgv8Kmhv6A183thOzGyamRWbWXFlZWUIMVtuy95j/GxNBV8ZM4j+3TtGHadFcrp04Lar8vj1up2U7D4SdRwRaUWRFQx3P+zuR4PPrwHpZpZNXY8it96qA4GdTexnrrsXunthTk5OqJmb67FFpaSnGvdMGBZ1lLi4a9xQsjLSeES9DJF2JbKCYWZ9LXift5mNDrLsA1YD+WY2xMwygJuAV6PK2VJle47wq/d28NUr8+jdJTPqOHHRIyuDO64ewmsffMz6nYeijiMirSTM22pfBN4GRphZhZndaWbTzWx6sMrfAB+a2TrgMeAmr1MNzABeBzYAL7v7+rByhm3OwlI6pqdy17ihUUeJqzuvHkLXzDTmLFAvQ6S9SAtrx+5+8zmWPw483siy14DXwsjVmjbsOszv3t/FjAnD6dW5Q9Rx4qpbx3TuunYY33l9I+9tP8io3O5RRxKRkEV9l1RSm7OghC6ZaXz9muTqXZxx+1V59MzK4OH5G6OOIiKtQAUjJO9XHGT+R7v5+jVD6dYpPeo4ocjqkMb0a4eyvHQvqzY355EbEUkkKhghmb2ghO6d0vna2Lyoo4Tq1ivyyOnSgYfnb8S90bufRSQJqGCEYM3W/SzdWMn0a4fRJTM5exdndMxI5d7xw3hn835WbNoXdRwRCZEKRggenl9CducMvnrl4KijtIqbxwyif7dM9TJEkpwKRpyt2LSXFZv2cff44XTKCO0mtDalQ1oqMybms3bbQZZubJtP24tIy6lgxJG7M3t+CX27ZvK3YwZFHadVfalwILk9OzJ7QYl6GSJJSgUjjpaV7qV46wHunTiczPTUqOO0qvTUFGZOKuCDHYeY/9HuqOOISAhUMOKkrnexkQHdO3JjYe65N0hCN4zqz9DsLGbPL6G2Vr0MkWSjghEnCzfsYV3FIWZOyicjrX3+saalpnB/UQEbdx/hdx/sijqOiMRZ+/yXLc5qa53ZC0rI69WJL1yW8EN3tMhnL+rHiD5dmLOwhOqa2qjjiEgcqWDEwe8//JgNuw5z/+QC0lLb9x9pSorxQFE+5ZXH+PV7jb6VXkQSUPv+1y0OamqdOQtLyO/dmc9d0j/qOG3Cpy/sy4X9u/LoolKq1MsQSRoqGC30m3U7KdtzlAeKCkhNaWiwwPbHzHhwSgHb9h/n52sqoo4jInGigtEC1TW1PLKwhAv6dWXqhX2jjtOmTBjRm0sHdee7i0o5VV0TdRwRiQMVjBZ4Ze0Otuw7zqyiAlLUu/gzZsaDRSPYeegkL63aHnUcEYmDMEfcm2dme8zsw0aW/62ZvR9MK8zsknrLtpjZB2b2npkVh5WxJU5X1/LoolIuGdiNyRf0jjpOmzR2eC9GD+nJE0vKOFmlXoZIoguzh/EsMLWJ5ZuBa939YuDbwNyzlk9w91HuXhhSvhb5afF2dhw8wawpIwiGJpez1PUyCthz5BQ/Xrk16jgi0kKhFQx3XwY0OqqOu69w9wPB7EpgYFhZ4u1kVQ2PLy7l8rwejMvPjjpOmzZmaC+uyc/me0s3cexUddRxRKQF2so1jDuB39ebd2C+ma0xs2lNbWhm08ys2MyKKytb502pL7yzjd2HTzGrSL2LWMwqKmD/sdM8u2JL1FFEpAXOWTDMLM/MMoLPV5vZPWbWNV4BzGwCdQXjn+o1j3X3y4DrgHvNbFxj27v7XHcvdPfCnJyceMVq1PHT1Ty5tIyrhvXiymG9Qj9eMrh0UA8mfaI3c5eVc/hkVdRxRKSZYulh/ApwMxsGPA9cAPwkHgc3s4uBp4Hr3f2Pw7W5+87g5x7gl8DoeBwvHp5/eyt7j57mwSkFUUdJKA8UFXDoRBXPLN8cdRQRaaZYCkatu1cBXwAecff7gBa/MMnMBgGvALe6e0m99iwz63LmMzAFaPBOq9Z25GQV339jE+NH5PCpwT2jjpNQPjmgG9d9si/z3tzMgWOno44jIs0QS8GoNrMvAbcCvw3azjlQtZm9CLwNjDCzCjO708ymm9n0YJV/BXoB3zvr9tk+wJtmtg5YBfzO3f/7PL5TaH741hYOHq9iVpF6F83xQFEBR09XM3d5edRRRKQZYhlD9A7gHuA/3b3czIYAL55rI3e/+RzL/w74uwbay4FL/nKLaB06XsVTy8uZMrIPFw/sHnWchFTQpwufu7g/z761hTuvHkJ25w5RRxKR83DOHoa7f+ju97j7j82sG9DR3f+tFbK1KU8tL+fIyWoeUO+iRe6fnM+p6hq+v3RT1FFE5DzFcpfUIjPramY9gA+An5jZd8KP1nbsO3qKH761mc9e3I8L+sXtBrF2aWhOZ75w2UB+tHIruw+fjDqOiJyHWK5h9HT3w9Rd9H7O3UcBnw43Vtvyg2XlnKiq4f7J6l3Ew8xJ+dTUOk8sKYs6ioich1gKRpqZ5QBfAn4Tcp42Z8+Rkzz/9hZuGDWA4b07Rx0nKeT27MSXL8/lxVXbqDhwPOo4IhKjWArGvwFvANvdfZWZDaXuPVDtwveWbKKqxpk5OT/qKEllxoThGMbji9XLEEkUsVz0fsndR7r714P5cne/Pvxo0dt58AQ/eWcbX/rUQAb3yoo6TlLp370jXxkziJ+tqWDL3mNRxxGRGMRy0bu/mb1sZruC6adm1i7GIn08OMd+3yT1LsJwz4RhpKcajy0qjTqKiMQgllNSPwQWAHnBtCBoS2rb9h3n5dXbuWl0LgO6d4w6TlLq3SWT267M41fv7aBsz5Go44jIOcRSMPq4+1PufiqYnqbuaeyk9tjiUlJTjHsnDI86SlK769phdExPZc5C9TJE2rpYCsZ+M7vJ/uRGmhjnIhmUVx7llbUV3HrFYPp0zYw6TlLrmZXB18YO4Xfv72LDrsNRxxGRJsRSMO4AvgrsBSqpe6fUHWGGitojC0vJTE9l+vhhUUdpF75+zVC6ZKYxZ0HJuVcWkcjEcpfUFnf/K3fv5e7Z7v5Z4LOtkC0SGz8+wm/e38ntV+XpXUetpFundL5+zVDmf7Sb9ysORh1HRBrR3BH3/jGuKdqQRxaW0DkjjWnjhkYdpV352tg8enRKZ7Z6GSJtVnMLRlKOS/rhjkP8/sOPuePqIXTvlBF1nHalS2Y6d107jKUbK1mzNakvkYkkrOYWDI9rijZizoISunVM585rhkQdpV366pWDye6cwcPz1csQaYsaLRhmdsDM9jcwHQBienDPzOaZ2R4za3DEvOCuq8fMrMzM3jezy+otu83MSoPptvP+Zudp7bYDLPrDHqaNG0rXzHOODyUh6JSRxj3jh7Ni0z5WbNobdRwROUtTPYxsIKeBKRuI9Wrws8DUJpZfB+QH0zTgSQAz6wl8CxhD3Xje3wperx6aOQtK6JWVwe1X5YV5GDmHr4wZRN+umcyeX4J7UnZkRRJWowXD3WuammLZubsvo+lnNq4Hnvc6K4HuZtaPutenL3D3/e5+gLqny5sqPC3yTvk+lpfu5e7xw8jqEMsghBKWzPRUZkwcTvHWAywrVS9DpC1p7jWMeBkAbK83XxG0NdYed+7OwwtK6N2lA7dcMTiMQ8h5+nJhLgN7dGT2/I3qZYi0IVEXjIbutvIm2v9yB2bTzKzYzIorKyvPO8CRU9VU19QyY+JwMtNTz3t7ib+MtBS+MTGfdRWHWLhhT9RxRCQQdcGoAHLrzQ8EdjbR/hfcfa67F7p7YU5OznkH6JqZzi/uvoq/HaPeRVvyhcsGkNerE7MXlFBbq16GSFvQrLukzCxeN8q/Cnw1uFvqCuCQu+8CXgemmFmP4GL3lKAtFGZGakpSPlqSsNJSU7h/cgEbdh3m9x9+HHUcEQGausKb3dKdm9mLwHgg28wqqLvzKR3A3b8PvAb8FVAGHAe+Fizbb2bfBlYHu3rI3fU0VzvzuUv688SSMuYsLGHqJ/uqqItErNGCcfadUMGtrvVf3drgKaKz9nHzOZY7cG8jy+YB8851DEleqSnGA0UF3PPCWn6zbic3XBrKfQ8iEqNYRtz7jJmVUHdd4Z3g5+Kwg4kATL2wLxf068ojC0uorqmNOo5IuxbLRe9/A8YCG909l7pnJJaGGUrkjJQU48GiArbsO84ra3dEHUekXYulYFS7eyWQYmbm7guAy861kUi8TLqgN5fkdufRRaWcrlYvQyQqsRSMQ2aWBbwJPG9mDwP6Wyutxqyul7Hj4Al+Wrz93BuISChiKRg3ACeB+6k7FbWDJB5ASdqma/KzuTyvB48vLuVkVUxvphGROIulYPxz8P6oKnd/xt1nA7PCDiZSn5kxq2gEuw+f4oV3tkUdR6RdiqVgNPTSv8/EO4jIuVw5rBdjh/fiyaVlHD9dHXUckXanqSe97zKzd4ERZra23lQKfNR6EUX+ZFbRCPYePc1zK7ZGHUWk3WnqSe+XgUXA/wO+Wa/9iLvrjXASiU8N7sGEETn8YNkmbrliEF002JVIq2lqPIwD7l7m7l8COgJFwXT+b/gTiaNZRSM4eLyKH761JeooIu1KLE9630tdb2NQML1sZveEHUykMRcN7MaUkX14ank5h45XRR1HpN2I5aL3XcBod/8Xd/8X6oZNnR5uLJGmzZpSwNFT1Ty1vDzqKCLtRiwFw4D6v8ZV0fAARyKt5hN9u/KZi/ox763N7Dt6Kuo4Iu1CU3dJnbkg/iNgpZn9TzP7n8AK4LnWCCfSlPsnF3CyqoYfLFMvQ6Q1NNXDWAXg7v8JTKNuvIoTwHR3/69WyCbSpOG9O3PDpQN4/u0t7Dl8Muo4IkmvqYLxx9NO7r7a3We7+8PuvrqJbURa1cxJ+VTVON9buinqKCJJr6nnMHLMrNFXgASvCGmSmU0FHgVSgafd/d/PWj4HmBDMdgJ6u3v3YFkN8EGwbJu7f/5cx5P2Z3CvLL5cOJCfvLONaeOG0r97x6gjiSStpnoYqUBnoEsjU5PMLBV4ArgOGAncbGYj66/j7g+4+yh3HwV8F3il3uITZ5apWEhTZkzMB+C7i8siTiKS3JrqYexy94dasO/RQJm7lwOY2UvA9TT+WpGbqRvzW+S8DOjekZtG5/KTd7Zx97XDGNSrU9SRRJJSTNcwmmkAUH/wgoqg7S8PZDYYGMKfD/2aaWbFZrbSzG5oYRZJcvdOGE5qivHY4tKoo4gkraYKxqQW7ruhguONrHsT8HN3rz/QwSB3LwS+AjxiZsMaPIjZtKCwFFdWVrYssSSsPl0zufWKwbyytoJNlUejjiOSlJp6l9T+Fu67AsitNz8Q2NnIujcBL551/J3Bz3LqBm66tJGcc9290N0Lc3L0mqv2bPr4YWSmp/LoQvUyRMIQy5PezbUayDezIWaWQV1RePXslcxsBNADeLteWw8z6xB8zgbGoleqyzlkd+7A7Vfl8Zv3d7Lx4yNRxxFJOqEVDHevBmYArwMbgJfdfb2ZPWRm9e96uhl4yd3rn666ACg2s3XAEuDf3V0FQ85p2rihdM5IY86CkqijiCSdpu6SajF3fw147ay2fz1r/n83sN0K4KIws0ly6t4pgzuvGcIjC0v5cMchPjmgW9SRRJJGmKekRCJxx9VD6NYxndnqZYjElQqGJJ2umencde1QFv9hD2u3HYg6jkjSUMGQpHTblXn0ysrQtQyROFLBkKSU1SGNu8cPY3npXt4p3xd1HJGkoIIhSeuWKwbTu0sHHp5fwp/fhCcizaGCIUkrMz2VGROHs2rLft4s2xt1HJGEp4IhSe3Gy3Pp3y1TvQyROFDBkKTWIS2Vb0zK573tB1mycU/UcUQSmgqGJL0vfmogg3p2Ui9DpIVUMCTppaemcP/kfNbvPMzr6z+OOo5IwlLBkHbh+lEDGJaTxewFJdTUqpch0hwqGNIupKYY908uoGT3UX77fmNv2ReRpqhgSLvxmYv68Ym+XXh0YSnVNbVRxxFJOCoY0m6kpBgPFBVQvvcYv3x3R9RxRBKOCoa0K1NG9uGiAd14bHEpp6vVyxA5HyoY0q6YGbOmFLB9/wl+tmZ71HFEEkqoBcPMpprZRjMrM7NvNrD8djOrNLP3gunv6i27zcxKg+m2MHNK+zK+IIfLBnXn8cVlnKyqiTqOSMIIrWCYWSrwBHAdMBK42cxGNrDqT919VDA9HWzbE/gWMAYYDXzLzHqElVXaFzPj76eMYNehk7y0alvUcUQSRpg9jNFAmbuXu/tp4CXg+hi3/TSwwN33u/sBYAEwNaSc0g5dNTybK4b25PElmzhxWr0MkViEWTAGAPVPElcEbWf7opm9b2Y/N7Pc89wWM5tmZsVmVlxZWRmP3NJOPDhlBHuPnuJHK7dEHUUkIYRZMKyBtrMfsf0NkOfuFwMLgefOY9u6Rve57l7o7oU5OTnNDivtz+V5PRlXkMOTSzdx9FR11HFE2rwwC0YFkFtvfiDwZ4/Yuvs+dz8VzD4FfCrWbUXiYVZRAQeOV/HsW5ujjiLS5oVZMFYD+WY2xMwygJuAV+uvYGb96s1+HtgQfH4dmGJmPYKL3VOCNpG4GpXbnckX9GHusnIOnaiKOo5ImxZawXD3amAGdf/QbwBedvf1ZvaQmX0+WO0bZrbezNYB3wBuD7bdD3ybuqKzGngoaBOJu1lFBRw+Wc0zy8ujjiLSplkyjQ9QWFjoxcXFUceQBHTPC2tYVrKXZf84gZ5ZGVHHEWk1ZrbG3QtjWVdPeosAD0wu4Njpan6wbFPUUUTaLBUMESC/Txeuv6Q/z6/YSuWRU+feQKQdUsEQCcycXMDpmlqeXKpehkhDVDBEAkOys/jiZQP48Ttb2XXoRNRxRNocFQyReu6bmI+788SSsqijiLQ5Khgi9eT27MSNl+fy09Xb2b7/eNRxRNoUFQyRs8yYkI+Z8d3FpVFHEWlTVDBEztK3Wya3jBnML9buYPPeY1HHEWkzVDBEGnD3+GFkpKbw6MKSqKOItBkqGCINyOnSga9eNZhfr9tJ6e4jUccRaRNUMEQaMX3cMLIy0nhkoa5liIAKhkijemRlcMfYPH73wS7W7zwUdRyRyKlgiDThzmuG0jUzjTkL1MsQUcEQaUK3julMGzeUhRt28972g1HHEYmUCobIOdw+dgg9OqUze4HumJL2TQVD5Bw6d0jj7vHDWFZSyeotGsdL2q9QC4aZTTWzjWZWZmbfbGD5LDP7yMzeN7NFZja43rIaM3svmF49e1uR1nTrFXnkdOnAw/M3Rh1FJDKhFQwzSwWeAK4DRgI3m9nIs1Z7Fyh094uBnwP/WW/ZCXcfFUyfRyRCHTNSuXf8MFaW72e6Rx0FAAAJ2klEQVRF2d6o44hEIswexmigzN3L3f008BJwff0V3H2Ju595w9tKYGCIeURa5KbRg+jXLZP/mr+RZBraWCRWYRaMAcD2evMVQVtj7gR+X28+08yKzWylmd3Q2EZmNi1Yr7iysrJliUWakJmeyoyJw1m77SBLS/T/mrQ/YRYMa6CtwV/LzOwWoBD4Tr3mQcHA5F8BHjGzYQ1t6+5z3b3Q3QtzcnJamlmkSV/6VC65PTsye36JehnS7oRZMCqA3HrzA4GdZ69kZpOB/wF83t3/OJiyu+8MfpYDS4FLQ8wqEpOMtBS+MTGfD3YcYv5Hu6OOI9KqwiwYq4F8MxtiZhnATcCf3e1kZpcCP6CuWOyp197DzDoEn7OBscBHIWYVidlfXzqAodlZzFlQQm2tehnSfoRWMNy9GpgBvA5sAF529/Vm9pCZnbnr6TtAZ+BnZ90+ewFQbGbrgCXAv7u7Coa0CWmpKcycnM8fPj7C7z7YFXUckVZjyXQetrCw0IuLi6OOIe1Aba0z9dFl1NQ68x+4ltSUhi7ZibR9ZrYmuF58TnrSW6QZUlKMWUUFbKo8xq/f2xF1HJFWoYIh0kyfvrAvF/bvyiMLS6mqqY06jkjoVDBEmsmsrpexbf9xfrGmIuo4IqFTwRBpgYmf6M2o3O48tqiUU9U1UccRCZUKhkgLmBkPTilg56GT/HT19nNvIJLAVDBEWujq4dmMHtKTxxeXcbJKvQxJXioYIi1kZjxYVMCeI6f48cqtUccRCY0KhkgcjBnai6uHZ/Pk0k0cO1UddRyRUKhgiMTJrCkF7Dt2mufe3hJ1FJFQqGCIxMllg3ow8RO9+cEb5Rw+WRV1HJG4U8EQiaNZRQUcOlHFvDc3Rx1FJO5UMETi6JMDujH1wr48s3wzB4+fjjqOSFypYIjE2QNFBRw9Xc3cZeVRRxGJKxUMkTgb0bcLn7u4P8+u2MLeo6fOvYFIglDBEAnBzMn5nKyq4ftLN0UdRSRuQi0YZjbVzDaaWZmZfbOB5R3M7KfB8nfMLK/esn8O2jea2afDzCkSb8NyOvPXlw7kRyu3svvwyajjiMRFaAXDzFKBJ4DrgJHAzWY28qzV7gQOuPtwYA7wH8G2I6kb0vVCYCrwvWB/Iglj5qR8amqdJ5aURR1FJC7C7GGMBsrcvdzdTwMvAdeftc71wHPB558Dk8zMgvaX3P2Uu28GyoL9iSSMQb068aXCXF5atZ0dB09EHUekxdJC3PcAoP7rOyuAMY2t4+7VZnYI6BW0rzxr2wHhRRUJx30Th/OLNRX89RNv0a1jetRxJEn16JTBy9OvDP04YRaMhgY5PnsA8cbWiWXbuh2YTQOmAQwaNOh88omErn/3jnz7hgt5o6Qy6iiSxLpmts4vI2EWjAogt978QGBnI+tUmFka0A3YH+O2ALj7XGAuQGFhYYNFRSRKN14+iBsv1y8zkvjCvIaxGsg3syFmlkHdRexXz1rnVeC24PPfAIvd3YP2m4K7qIYA+cCqELOKiMg5hNbDCK5JzABeB1KBee6+3sweAord/VXgGeBHZlZGXc/ipmDb9Wb2MvARUA3c6+4amUZEJEJW9wt9cigsLPTi4uKoY4iIJAwzW+PuhbGsqye9RUQkJioYIiISExUMERGJiQqGiIjERAVDRERiklR3SZlZJbC1mZtnA3vjGCdKyfJdkuV7gL5LW5Qs3wNa9l0Gu3tOLCsmVcFoCTMrjvXWsrYuWb5LsnwP0Hdpi5Lle0DrfRedkhIRkZioYIiISExUMP5kbtQB4ihZvkuyfA/Qd2mLkuV7QCt9F13DEBGRmKiHISIiMWn3BcPMpprZRjMrM7NvRp2nucxsnpntMbMPo87SUmaWa2ZLzGyDma03s5lRZ2ouM8s0s1Vmti74Lv8n6kwtYWapZvaumf026iwtYWZbzOwDM3vPzBL6jaVm1t3Mfm5mfwj+zoQ29F67PiVlZqlACVBE3aBNq4Gb3f2jSIM1g5mNA44Cz7v7J6PO0xJm1g/o5+5rzawLsAa4IUH/uxiQ5e5HzSwdeBOY6e4rz7Fpm2Rms4BCoKu7fzbqPM1lZluAQndP+OcwzOw5YLm7Px2MPdTJ3Q+Gcaz23sMYDZS5e7m7nwZeAq6POFOzuPsy6sYUSXjuvsvd1wafjwAbSNAx3b3O0WA2PZgS8rc0MxsIfAZ4OuosUsfMugLjqBtbCHc/HVaxABWMAcD2evMVJOg/TMnKzPKAS4F3ok3SfMFpnPeAPcACd0/U7/II8I9AbdRB4sCB+Wa2xsymRR2mBYYClcAPg1OFT5tZVlgHa+8FwxpoS8jf/pKRmXUGfgHc7+6Ho87TXO5e4+6jqBubfrSZJdwpQzP7LLDH3ddEnSVOxrr7ZcB1wL3BKd1ElAZcBjzp7pcCx4DQrsW294JRAeTWmx8I7Iwoi9QTnO//BfCCu78SdZ54CE4VLAWmRhylOcYCnw/O/b8ETDSzH0cbqfncfWfwcw/wS+pOTyeiCqCiXq/159QVkFC094KxGsg3syHBxaKbgFcjztTuBReKnwE2uPvsqPO0hJnlmFn34HNHYDLwh2hTnT93/2d3H+juedT9PVns7rdEHKtZzCwruJmC4PTNFCAh7y5094+B7WY2ImiaBIR2c0haWDtOBO5ebWYzgNeBVGCeu6+POFazmNmLwHgg28wqgG+5+zPRpmq2scCtwAfBuX+Af3H31yLM1Fz9gOeCO/JSgJfdPaFvSU0CfYBf1v1eQhrwE3f/72gjtch9wAvBL73lwNfCOlC7vq1WRERi195PSYmISIxUMEREJCYqGCIiEhMVDBERiYkKhoiIxEQFQ+Q8mFlN8IbTM1Pcnqo1s7xkeNuwJK92/RyGSDOcCF7zIdLuqIchEgfB+Ar/EYx9scrMhgftg81skZm9H/wcFLT3MbNfBuNkrDOzq4JdpZrZU8HYGfODp8NF2gQVDJHz0/GsU1I31lt22N1HA49T92ZXgs/Pu/vFwAvAY0H7Y8Ab7n4Jde/+OfOGgXzgCXe/EDgIfDHk7yMSMz3pLXIezOyou3duoH0LMNHdy4MXJ37s7r3MbC91g0FVBe273D3bzCqBge5+qt4+8qh7/Xl+MP9PQLq7/9/wv5nIuamHIRI/3sjnxtZpyKl6n2vQdUZpQ1QwROLnxno/3w4+r6Du7a4Af0vdEK0Ai4C74Y8DLHVtrZAizaXfXkTOT8d6b9AF+G93P3NrbQcze4e6X8RuDtq+Acwzs3+gbmS0M28SnQnMNbM7qetJ3A3sCj29SAvoGoZIHATXMArdfW/UWUTColNSIiISE/UwREQkJuphiIhITFQwREQkJioYIiISExUMERGJiQqGiIjERAVDRERi8v8Bui10VBE8YZYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "X = [[1,0,0],\n",
    "    [1,1,0],\n",
    "    [1,0,1],\n",
    "    [1,1,1]]\n",
    "\n",
    "y = [0, 1, 1, 1]\n",
    "\n",
    "w=perceptron_train(np.array(X),y)\n",
    "print('Learned Weights:', w)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Predict OR with The learned weights of the Perceptron"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1, 0, 0] 0  OR  0  =  0\n",
      "[1, 1, 0] 1  OR  0  =  1\n",
      "[1, 0, 1] 0  OR  1  =  1\n",
      "[1, 1, 1] 1  OR  1  =  1\n"
     ]
    }
   ],
   "source": [
    "for i in range(4):\n",
    "    print(X[i], X[i][1],' OR ', X[i][2],' = ', predict(X[i],w))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exercise\n",
    "## Re-train Perceptron to Preform logic AND\n",
    "* How many epoch's needed ?\n",
    "* what are the learned weights ?\n",
    "* Check weights by predicting logic AND "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Re-train Percepron to Preform logic AND\n",
    "Same perceptron,can learn to perform logic AND, When its trained on the correct data:\n",
    "\n",
    "X1 | X2 | AND\n",
    "--- | --- | ---\n",
    "*0* | 0 | **0**\n",
    "*1* | 0 | **0**\n",
    "*0* | 1 | **0**\n",
    "*1* | 1 | **1**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--------------------------\n",
      "epoch:  0\n",
      "0 x: [1 0 0] w: [0. 0. 0.] Wx:  0.0 y: 0 correct no update\n",
      "1 x: [1 1 0] w: [0. 0. 0.] Wx:  0.0 y: 0 correct no update\n",
      "2 x: [1 0 1] w: [0. 0. 0.] Wx:  0.0 y: 0 correct no update\n",
      "3 yhat<y X: [1 1 1] w: [0. 0. 0.] Wx: 0.0 y: 1 => W updated: [1. 1. 1.]\n",
      "--------------------------\n",
      "epoch:  1\n",
      "0 yhat>y X: [1 0 0] w: [1. 1. 1.] Wx: 1.0 y: 0 => W updated: [0. 1. 1.]\n",
      "1 yhat>y X: [1 1 0] w: [0. 1. 1.] Wx: 1.0 y: 0 => W updated: [-1.  0.  1.]\n",
      "2 x: [1 0 1] w: [-1.  0.  1.] Wx:  0.0 y: 0 correct no update\n",
      "3 yhat<y X: [1 1 1] w: [-1.  0.  1.] Wx: 0.0 y: 1 => W updated: [0. 1. 2.]\n",
      "--------------------------\n",
      "epoch:  2\n",
      "0 x: [1 0 0] w: [0. 1. 2.] Wx:  0.0 y: 0 correct no update\n",
      "1 yhat>y X: [1 1 0] w: [0. 1. 2.] Wx: 1.0 y: 0 => W updated: [-1.  0.  2.]\n",
      "2 yhat>y X: [1 0 1] w: [-1.  0.  2.] Wx: 1.0 y: 0 => W updated: [-2.  0.  1.]\n",
      "3 yhat<y X: [1 1 1] w: [-2.  0.  1.] Wx: -1.0 y: 1 => W updated: [-1.  1.  2.]\n",
      "--------------------------\n",
      "epoch:  3\n",
      "0 x: [1 0 0] w: [-1.  1.  2.] Wx:  -1.0 y: 0 correct no update\n",
      "1 x: [1 1 0] w: [-1.  1.  2.] Wx:  0.0 y: 0 correct no update\n",
      "2 yhat>y X: [1 0 1] w: [-1.  1.  2.] Wx: 1.0 y: 0 => W updated: [-2.  1.  1.]\n",
      "3 yhat<y X: [1 1 1] w: [-2.  1.  1.] Wx: 0.0 y: 1 => W updated: [-1.  2.  2.]\n",
      "--------------------------\n",
      "epoch:  4\n",
      "0 x: [1 0 0] w: [-1.  2.  2.] Wx:  -1.0 y: 0 correct no update\n",
      "1 yhat>y X: [1 1 0] w: [-1.  2.  2.] Wx: 1.0 y: 0 => W updated: [-2.  1.  2.]\n",
      "2 x: [1 0 1] w: [-2.  1.  2.] Wx:  0.0 y: 0 correct no update\n",
      "3 x: [1 1 1] w: [-2.  1.  2.] Wx:  1.0 y: 1 correct no update\n",
      "--------------------------\n",
      "epoch:  5\n",
      "0 x: [1 0 0] w: [-2.  1.  2.] Wx:  -2.0 y: 0 correct no update\n",
      "1 x: [1 1 0] w: [-2.  1.  2.] Wx:  -1.0 y: 0 correct no update\n",
      "2 x: [1 0 1] w: [-2.  1.  2.] Wx:  0.0 y: 0 correct no update\n",
      "3 x: [1 1 1] w: [-2.  1.  2.] Wx:  1.0 y: 1 correct no update\n",
      "--------------------------\n",
      "epoch:  6\n",
      "0 x: [1 0 0] w: [-2.  1.  2.] Wx:  -2.0 y: 0 correct no update\n",
      "1 x: [1 1 0] w: [-2.  1.  2.] Wx:  -1.0 y: 0 correct no update\n",
      "2 x: [1 0 1] w: [-2.  1.  2.] Wx:  0.0 y: 0 correct no update\n",
      "3 x: [1 1 1] w: [-2.  1.  2.] Wx:  1.0 y: 1 correct no update\n",
      "Learned Weights: [-2.  1.  2.]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAEKCAYAAAD9xUlFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3Xd8lYW9x/HPLyEQVphhj7D3CgG3dYvUiosl9ra9vaUi1tVbV2/bW1u7bLVVlF5bbW1BJYJaRGzrXlUkCXsjAgkrYe/9u3/kkMvFACHJc54zvu/X67w44yH5nhcvzu886/uYuyMiIgKQEnYAERGJHRoKIiJSRkNBRETKaCiIiEgZDQURESmjoSAiImU0FEREpIyGgoiIlNFQEBGRMjXCDnCmmjZt6llZWWHHEBGJK/n5+ZvdPfN0y8XdUMjKyiIvLy/sGCIiccXM1lRkOW0+EhGRMhoKIiJSRkNBRETKaCiIiEgZDQURESkT2FAws3Qz+9TM5pnZIjP7cTnL1DKzKWa20sxmmVlWUHlEROT0glxTOABc4u79gP7AEDM7+4Rlvglsc/fOwKPALwPMIyIipxHYeQpeep3P3ZGHaZHbidf+HAb8d+T+VGCCmZnrGqGntHTjTmbO3xB2jOphxtA+LejeIiPsJCJCwCevmVkqkA90Bp5w91knLNIaKARw98NmtgNoAmw+4eeMBcYCtGvXLsjIceEHryxk9uptmIWdpOrcYfIna3jt9gto0SA97DgiSS/QoeDuR4D+ZtYQeNnMerv7wuMWKe9j7QtrCe7+FPAUQE5OTlKvRawq2c3s1du4d0h3xl3UKew4VbayeBfXTPiI8c8V8MLYs0lL1bEPImGKyv9Ad98OvAsMOeGlIqAtgJnVABoAW6ORKV7l5hWRmmLckN067CjVonOz+vzyhr7kr9nGz2cuDTuOSNIL8uijzMgaAmZWG7gMOPF//XTga5H7NwJva3/CyR0+cpRpBUVc3C2TZhmJs6nlK/1a8fVzs3jmo895LVH2lYjEqSDXFFoC75jZfGA28Ia7zzCzB83smsgyTwNNzGwlcDdwX4B54t67y0oo2XWAETltw45S7R4Y2oMB7Rpyz9R5fFay+/R/QUQCYfH2xTwnJ8eTtSV17F/yKFi7nY/vvyQht72v376Pqx//kKb1avLK+POoUzPuSnxFYpaZ5bt7zumWS7xPlgRVsusAby8t5obs1gk5EABaNazN70b1Z0Xxbr7/8kLi7QuLSCJIzE+XBPRSQRGHjzrDE3DT0fEu6JLJXZd15eU565g8a23YcUSSjoZCHHB3cvMKGdi+EZ2b1Qs7TuBuu7gzF3XL5MFXFzOvcHvYcUSSioZCHChYu43PSvYwMsHXEo5JSTEeHdGfzPq1uHVyAdv2HAw7kkjS0FCIA7mzi6hTM5WhfVuGHSVqGtWtyZNjsinZdYC7cudy9Kj2L4hEg4ZCjNtz4DAz5q/n6r4tqVcruY7G6de2IT/4Sk/eXVbChHdWhh1HJCloKMS41xZsYM/BIwl5bkJF3HxWO67t34pH31zOBytKwo4jkvA0FGJc7uxCOmbWZWD7RmFHCYWZ8bPr+9ClWT3ueGEu67fvCzuSSELTUIhhn5XsJm/NNkbktMUSoRK1kurUrMHEmwdy4NARxj9XwMHDR8OOJJKwNBRiWG5eIakpxvUJUn5XFZ0y6/GrG/sxZ+12fjZzSdhxRBKWhkKMOnTkKNPy13Fxt2Y0q5845XdV8eW+LfnGeVn8+V+reXXe+rDjiCQkDYUY9e6yEjbvPsDIQcm5g/lk7r+qB9ntGnLftPmsLFZxnkh101CIUbl5hTStV4uLumWGHSWm1KyRwhNjsqmVlsq4SfnsOXA47EgiCUVDIQYV79pfWn43MHHL76qiZYPaPDZqACtLdnP/SwtUnCdSjfSJE4NeKljHkaOetOcmVMT5XZpy92VdmT5vPX/9ZE3YcUQShoZCjDlWfpfTvhGdMhO//K4qxl/cmYu7ZfKTGYuZs3Zb2HFEEoKGQozJX7ONVSV7GKEdzKeVkmI8OrI/zeqnM35yAVtVnCdSZRoKMSY3r5C6NVP5cp/kKb+rioZ1ajLx5mw27z7InVPmckTFeSJVoqEQQ3YfOMyM+Ru4um8r6iZZ+V1V9G3TkB9d05P3l5fw+Nsrwo4jEtc0FGLIzPkb2HvwCCMGtQk7Sty5aXA7rh/Qmt+9tYL3lqs4T6SyNBRiyJS8Qjpl1iW7XXKW31WFmfHQdX3o2qw+d74wh3UqzhOpFA2FGLGyeDf5Kr+rkto1U5l4czaHjjjjJ6s4T6QyNBRixItl5XfadFQVHTPr8asb+zK3cDsPvbY47DgicUdDIQYcOnKUaQXruKR7MzLr1wo7Ttwb2qcl3zy/A89+vIbpKs4TOSOBDQUza2tm75jZEjNbZGZ3lLPMRWa2w8zmRm4/DCpPLHtnaXFp+Z3OYK42913VnZz2jbhv2nxWbNoVdhyRuBHkmsJh4Lvu3gM4GxhvZj3LWe4Dd+8fuT0YYJ6YlZtXRGZ9ld9Vp7TUFCbclE2dmqncMimf3SrOE6mQwIaCu29w94LI/V3AEkBXizlB8c79vLOsmBuy21BD5XfVqkWDdB4bNYDPN+/hvmnzVZwnUgFR+RQysyxgADCrnJfPMbN5Zva6mfU6yd8fa2Z5ZpZXUpJYx6BPKyu/0w7mIJzbuSnfvaIbM+Zv4Nl/rQ47jkjMC3womFk9YBpwp7vvPOHlAqC9u/cDHgdeKe9nuPtT7p7j7jmZmYmzicXdeTGvkEFZjeio8rvAjPtSJy7t3oyHZi6hQMV5IqcU6FAwszRKB8Jkd3/pxNfdfae7747cnwmkmVnTIDPFkrw121i1eY8qsgOWkmI8MqI/zTNKi/O27D4QdiSRmBXk0UcGPA0scfdHTrJMi8hymNngSJ4tQWWKNbmzS8vvhqr8LnAN6qQxccxAtuxRcZ7IqQS5pnAe8FXgkuMOOR1qZreY2S2RZW4EFprZPOAxYJQnyd7A3QcO89qCDXyln8rvoqVPmwb8+JpefLBiM797S8V5IuUJ7NPI3T8ETtnX4O4TgAlBZYhlr81fz96DRxiuTUdRNWpQW/JWb+Pxt1eQ3a4hF3VrFnYkkZiiYyBDMmV2IZ2b1SO7XcOwoyQVM+On1/amW/P63DllLkXb9oYdSSSmaCiEYGXxLgrWbmdEThuV34WgtDhvIEcixXkHDh8JO5JIzNBQCEFuXhE1VH4Xqg5N6/Lw8L7MK9rBT2csCTuOSMzQUIiyQ0eO8lJBEZf2aEbTeiq/C9OQ3i351gUd+Osna/jb3HVhxxGJCRoKUfb20mI27z6ocxNixD1DujMoqxH3TVvAchXniWgoRNuLeYU0q1+LL3VNnDOz49mx4ry6tWqoOE8EDYWoKi2/K+GGgSq/iyXNM9J5fPQAVm/ew71TVZwnyU2fTFE0taAoUn6nTUex5pxOTfjPK7vx2oIN/Omj1WHHEQmNhkKUlJbfFTE4qzEdmtYNO46U45YLO3FZj2b8bOYS8tdsDTuOSCg0FKJk9uptfL55DyMGaS0hVqWkGL8Z3p+WDdMZP3kOm1WcJ0lIQyFKcvMKqVerBkP7tAg7ipzCseK8rXsPcscLc1ScJ0lHQyEKdu0/xGvzN/CVfi2pU1Pld7Gud+sGPHhNLz5auYXfvrk87DgiUaWhEAWvzd/AvkMqv4snIwe1ZfjANjz+9kreWVocdhyRqNFQiIIpeYV0aVaPAW1VfhcvzIyfXNubHi0zuHPKXAq3qjhPkoOGQsBWbNrFnLXbGZHTVuV3cSY9LZWJY7I5etQZ/5yK8yQ5aCgELDevkBopxnXZrcOOIpWQ1bQuvx7Rj/lFO3jw1cVhxxEJnIZCgErL79ZxWY/mKr+LY1f2asG3L+zI5FlreamgKOw4IoHSUAjQW0uK2bLnICMGqSI73n3vym4M7tCYB15ewNKNO8OOIxIYDYUA5eYV0jyjFhd2UfldvKuRmsKE0QOoVyuNcZMK2LX/UNiRRAKhoRCQTTv38+6yYm7IVvldomiWkc6Emwawdute7lFxniQofVoFZGp+EUcdld8lmLM7NuF7V3bj9YUbefrDz8OOI1LtNBQCUFp+V8jgDo3JUvldwvn2hR25vGdzfvH6UvJWqzhPEouGQgA+/Xwrq7fsZaTWEhKSmfHr4f1o3ag2458rUHGeJJTAhoKZtTWzd8xsiZktMrM7ylnGzOwxM1tpZvPNLDuoPNGUm1dEvVo1uErldwmrQe00nhyTzfa9h7j9eRXnSeIIck3hMPBdd+8BnA2MN7OeJyxzFdAlchsLTAwwT1Ts2n+ImQs28JV+rVR+l+B6tWrAT4b15l+fbeGRN5aFHUekWgQ2FNx9g7sXRO7vApYAJ57WOwz4i5f6BGhoZi2DyhQNMyLldyNydG5CMhgxqC0jctrwxDuf8daSTWHHEamyqOxTMLMsYAAw64SXWgOFxz0u4ouDI65MmV1I1+b16K/yu6Tx4LDe9GyZwV0qzpMEEPhQMLN6wDTgTnc/8VTQ8hrivrBx1szGmlmemeWVlJQEEbNaLN+0i7mFKr9LNulpqUy8ORsHxk3OZ/8hFedJ/Ap0KJhZGqUDYbK7v1TOIkXA8YfotAHWn7iQuz/l7jnunpOZGbtnB+fOLiQt1bhuQFyv7EgltG9Sl0dG9Gfhup38WMV5EseCPPrIgKeBJe7+yEkWmw78W+QopLOBHe6+IahMQTp4+Cgvzyktv2ui8rukdHnP5tzypU48/+lapuarOE/iU5CHx5wHfBVYYGZzI889ALQDcPffAzOBocBKYC/wjQDzBOrtpZtKy+90bkJS+88rujK3cBvff3kBvVpl0KNlRtiRRM5IYEPB3T+k/H0Gxy/jwPigMkTTlNmFtMhI58Kusbt5S4JXIzWFx0YP4OrHPmTcpHymf+d8MtLTwo4lUmE6o7kabNyxn/eWl3DDwNakpmgHc7JrVj+dCTdlU7htH/e8qOI8iS8aCtVgWkFp+d3wgdp0JKUGd2jMvUO68fdFG/njByrOk/ihoVBF7k5uXiFnqfxOTvCtCzpyZa/m/OLvS/n0cxXnSXzQUKiiWZ9vZc2WvYwcpLUE+f/MjIeH96Nto9rc9lwBxbv2hx1J5LROOxTMLMvMakbun29mt5qZDqmIyM0rpH6tGlzVO67bOSQgGelpPDlmIDv2lRbnHT5yNOxIIqdUkTWFVwA3s07AX4AewHOBpooTO4+V3/VvRe2aqWHHkRjVs1UGD13Xh09WbeU3bywPO47IKVVkKBx190PA9cBv3f07xHk/UXWZMW8D+w8d1XUT5LRuHNiG0YPbMvHdz3hjsYrzJHZVZCgcNrPhlJ6INiPynA68BqbkFdKteX36tmkQdhSJAz/6Si96t87g7ty5rN2i4jyJTRUZCv8OXAz8yt1XmVkH4PlgY8W+ZRt3Ma9wOyMGqfxOKiY9LZWJYwZiqDhPYtdph4K7L3T3W919kpk1AGq7+0NRyBbTcvNUfidnrm3jOjw6sj+L1u/kR39bFHYckS+oyNFHb5lZhpk1AhYAz5nZw8FHi13Hyu8u79mcxnVrhh1H4sylPZpz60WdmJJXSG5e4en/gkgUVWTzUePIdRCuB5519/7AlcHGim1vLdnE1j0HGa4dzFJJd1/elXM6NuEHryxk0fodYccRKVORoVDDzDKB4cCrAeeJC1PyIuV3XVR+J5VzrDivQe00bp1cwI59h8KOJAJUbCg8BLwHFLr7p2bWEUjaMpcNO/bx/vISbhzYRuV3UiWZ9WvxxJhsirbt43svzlNxnsSEiuxofsHde7r7tyKPV7n7sOCjxaZp+ZHyu5w2YUeRBDAoqzH3X9Wdfy7exFPvrwo7jkiFdjS3MrNcM9sQuU0xs1bRCBdrjh51cvOKOLtjY9o3UfmdVI9vnt+Bq3q34Ff/WMasVVvCjiNJriKbj/4EvAFkRW5vRJ5LOrM+38rarSq/k+plZvzqxr60b1yH256fQ/FOFedJeCoyFJq7+x/c/UDk9kegedDBYtGLkfK7Ib1UfifVq356Gk/enM2u/Ye4TcV5EqKKDIWtZjbK/s9IIOnK4XfuP8TMhRu4RuV3EpDuLTL42XV9+PTzrTz8z2Vhx5EkVdGai38DNgMllHYg/XuQoWLRq/PWl5bfadORBOj67DbcdFY7/ue9Vfxj0caw40gSqsjRR6vdfai7N3H3pu5+NXB1FLLFlNzZhXRvUZ8+rVV+J8H64dU96dO6Af+ZO4/Vm/eEHUeSTGWvvHZPtaaIcUs37mRe0Q5G5Kj8ToKXnpbKk2OySUkxxk0uUHGeRFVlh0JSfTLmzi4iLdW4VuV3EiWlxXn9WLJhJz94ZWHYcSSJVHYoJM2pl6Xld0Vc0bOFyu8kqi7p3pzbLu7Mi/lFTJm9Nuw4kiROOhTMbJuZbS3ntg047clrZvaMmRWbWblfc8zsIjPbYWZzI7cfVuF9BObNJZvYtveQzmCWUNx1eVfO69yEH/xtEQvXqThPgneqNYWmQGY5t6ZArQr87D8DQ06zzAfu3j9ye7ACPzPqpswupGWDdC5Q+Z2EIDXF+N2oATSuU1PFeRIVJx0K7n7kVLfT/WB3f584P59h/fZ9vL9C5XcSrqb1Sovz1m/fx3dz53H0aNJsvZUQVHafQnU5x8zmmdnrZtYr5CxfMC2/CHcYPlDnJki4BrZvxANDe/Dmkk38j4rzJEBhDoUCoL279wMeB1452YJmNtbM8swsr6SkJCrhjh51Xswv4pyOTWjXpE5UfqfIqXzjvCy+3LclD/9jKR9/puI8CUZoQ8Hdd7r77sj9mUCamTU9ybJPuXuOu+dkZkZn2/4nn29R+Z3EFDPjlzf0JatpXb6j4jwJSKWOPjKzKu8rMLMWFjkTzMwGR7LEzNefF/OKqJ9egyG9W4QdRaRMvVo1+P3NA9lz4DC3PTeHQyrOk2pW2aOPTvt13cyeBz4GuplZkZl908xuMbNbIovcCCw0s3nAY8Aoj5FLT+3Yd4iZCzYwrH8r0tNUfiexpWvz+vz8+j58unorD/9DxXlSvWqc7IUTjzAys8ZA+nFPrT/VD3b30ad5fQIwoQIZo+7Vees5cPgoI3PahR1FpFzXDmhN3pqtPPX+KrLbNWRIb9W5S/WoyJXXvmxmy4EiYFbkz7eDDham3LzS8rverTPCjiJyUj+4uif92jTgey/O53MV50k1qciO5oeA84Bl7t4WuBJ4N8hQYVqyYSfzi3YwcpDK7yS21aqRyhNjsklNNcZNymffQRXnSdVVZCgcdvcSIMXMzN3fALIDzhWa3LxCaqamcG1/ld9J7GvTqA6/HdmfZZt28V+vLCRGdstJHKvIUNhhZnWBD4G/mNlvgIQ85OHA4SO8PGcdl/dqTiOV30mcuKhbM75zSRemFRTxwuzCsONInKvIULgW2A/cSelmo3Uk6EV23lxczPa9hxiRo3MTJL7ccWkXLujSlB9NV3GeVE1FhsL9kb6jQ+7+tLs/AtwddLAwTMkrpFWDdM7vXO45dCIx61hxXpO6NbllUj479qo4TyqnIkOhvKbTL1d3kLCt376PD1R+J3Gscd2aPDEmm00793N37lwV50mlnOqM5m+b2RxKTz4rOO62AlgcvYjRMfVY+Z02HUkcy27XiO8P7cFbS4uZ+N5nYceROHTSk9eAXOAt4OfAfcc9v8vdiwNNFWWl5XeFnNupCW0bq/xO4tvXzs0if+12fvPPZQxo15BzO2lzqFTcqa6nsM3dV7r7cKA2cHnklnBXm/lk1RYKt+5T+Z0kBDPjF9f3oUPTutz+/Bw27lBxnlRcRc5oHk/pWkO7yC3XzG4NOlg05eYVkpFegyt7qfxOEkPdSHHe3oNHuO25AhXnSYVVZEfzt4HB7v6Auz8AnAXccpq/Ezd27DvE6ws3Mqx/a5XfSULpEinOy1uzjV+8vjTsOBInKjIUDDj++LZDkecSwvRj5XfadCQJaFj/1nztnPY8/eHnzFywIew4EgdOuqPZzGq4+2Hgr8AnZjYt8tJ1wLPRCBcNubML6dEyg16tVH4nien7X+7JvKId3DN1Pt1b1KdjZr2wI0kMO9WawqcA7v4rYCywF9gH3OLuv45CtsAtXr+TBet2MDKnjcrvJGHVrJHCE2OySUs1xk0qYO/Bw2FHkhh2qqFQ9inp7rPd/RF3/427z45Crqg4Vn43TOV3kuBaN6zN70YNYHnxLv7rZRXnycmd6jyFTDM7aZ1FpO4ibh04fIRX5q7jCpXfSZK4sGsmd1zahd++uYKBWY0Yc1b7sCNJDDrVUEgF6pFAO5WP98biTSq/k6Rz+yVdKFi7nR9PX0yf1g3o26Zh2JEkxpxqKGxw9wejliTKpswupHXD2pyn8jtJIikpxm9H9ufqxz5g3KQCXrv9fBrW0Zqy/J8K7VNINOu27+PDlZu5QeV3koQa163JkzcPpHjXfu6aouI8+f9ONRQujVqKKJuaVwTA8IFtQk4iEo7+bRvyg6t78s6yEp58d2XYcSSGnKr7aGs0g0TLsfK78zo1VfmdJLWvnt2ea/q14pE3lvPRys1hx5EYUZEzmhPKx6u2ULRtH8NztJYgyc3M+Pn1feiUWY/bn5/Dhh37wo4kMSDphoLK70T+T91aNZh480D2HzrC+MkFHDys4rxkF9hQMLNnzKzYzBae5HUzs8fMbKWZzTez7KCyHLNjb2n53bUDVH4nckznZvX45Y19KVi7nZ+/viTsOBKyINcU/kz5l/I85iqgS+Q2FpgYYBYAps9bx8HDR3VugsgJru7biq+fm8WfPlrNjPnrw44jIQpsKLj7+8CpdlYPA/7ipT4BGppZy6DyAEzJK6Rnywx6t24Q5K8RiUsPDO1BdruG3Dt1PiuLd4cdR0IS5j6F1kDhcY+LIs8FYtH6HSxct1MV2SIncaw4r1ZaKrdOzldxXpIKcyiUd9ZYuWfRmNlYM8szs7ySkpJK/bItuw/StXk9hvVvVam/L5IMWjaozWOjBrCieDcPvLRAxXlJKMyhUAQc/7W9DVDuxkx3f8rdc9w9JzOzcpeIvrBrJv+860s6pV/kNM7v0pS7LuvKK3PXM2nW2rDjSJSFORSmA/8WOQrpbGCHu+vSUCIx4LaLO3NRt0x+8upi5hVuDzuORFGQh6Q+D3wMdDOzIjP7ppndYmbHru88E1gFrAT+ANwaVBYROTMpKcajI/qTWb8Wt04uYNueg2FHkiixeNtmmJOT43l5eWHHEEkK8wq3M/z3H3Nu5yY887VBpKhAMm6ZWb6755xuuaQ7o1lEKq5f24b88Cs9eXdZCRPeUXFeMtBQEJFTGnNWO64b0JpH31zOBysqd/SfxA8NBRE5JTPjoet606VZPe54YS7rt6s4L5FpKIjIadWpWVqcd/DwUcY/p+K8RKahICIV0imzHr+6sS9z1m7nZzNVnJeoNBREpMKG9mnJv5/XgT//azXT56k4LxFpKIjIGbl/aHcGtm/EfdPms7J4V9hxpJppKIjIGUlLTeGJm7KpnZbKLZMK2HNAxXmJRENBRM5YiwbpPDZ6AKtKdnO/ivMSioaCiFTKeZ2bcvflXZk+bz1//WRN2HGkmmgoiEil3XpRZy7p3oyfzFjMnLXbwo4j1UBDQUQqLSXFeGREP5pnpDN+cgFbVZwX9zQURKRKGtapycQxA9m8+yB3TpnLkaPavxDPNBREpMr6tGnAf1/Ti/eXl/D42yvCjiNVoKEgItVi9OC2XJ/dmt+9tYL3lqs4L15pKIhItTAzHrq2D92a1+fOF+awTsV5cUlDQUSqTe2aqTw5JptDR5zxk1WcF480FESkWnXMrMevh/dlbuF2Hnptcdhx5AxpKIhItRvSuyX/cX4Hnv14DX+buy7sOHIGNBREJBD3XtWdQVmNuG/aAlZsUnFevNBQEJFApKWmMOGmbOrWSuWWSfnsVnFeXNBQEJHANM8oLc77fPMe7ps2X8V5cUBDQUQCdW6npnz3im7MmL+BZ/+1Ouw4choaCiISuHFf6sSl3Zvx0MwlFKg4L6YFOhTMbIiZLTOzlWZ2Xzmvf93MSsxsbuT2H0HmEZFwlBbn9adFg9LivC27D4QdSU4isKFgZqnAE8BVQE9gtJn1LGfRKe7eP3L7Y1B5RCRcDeqkMXHMQLbsUXFeLAtyTWEwsNLdV7n7QeAFYFiAv09EYlzv1g148JpefLBiM797S8V5sSjIodAaKDzucVHkuRPdYGbzzWyqmbUNMI+IxICRg9py48A2PP72Ct5dVhx2HDlBkEPBynnuxPXFV4Esd+8LvAk8W+4PMhtrZnlmlldSovZFkXhmZvxkWO/S4rwpcynatjfsSHKcIIdCEXD8N/82wPrjF3D3Le5+bI/TH4CB5f0gd3/K3XPcPSczMzOQsCISPbVrpvL7mwdyJFKcd+DwkbAjSUSQQ2E20MXMOphZTWAUMP34Bcys5XEPrwGWBJhHRGJIVtO6PDy8H/OKdvDTGfqvHysCGwrufhi4DfgHpR/2ue6+yMweNLNrIovdbmaLzGwecDvw9aDyiEjsGdK7BWMv7MhfP1FxXqyweDvtPCcnx/Py8sKOISLV5PCRo9z0h1ksWLeDv912Hl2b1w87UkIys3x3zzndcjqjWURCVSM1hQk3DaBurRoqzosBGgoiErpmGek8PnoAqzfv4d6pKs4Lk4aCiMSEczo14XtXdue1BRv400erw46TtDQURCRm3PKljlzeszk/m7mE/DVbw46TlDQURCRmmBm/Ht6P1o1qM37yHDarOC/qNBREJKY0qJ3Gk2Oy2bb3IHe8MEfFeVGmoSAiMadXqwb8ZFhvPlq5hd++uTzsOElFQ0FEYtKIQW0ZkdOGx99eyTtLVZwXLRoKIhKzHhzWm54tM7hzylwKt6o4Lxo0FEQkZqWnpTLx5myOujP+ORXnRYOGgojEtPZN6vKb4f2YX7SDB19dHHachKehICIx74peLfj2lzoyedZaXp5TFHachKahICJx4XtXdOOsDo25/6UFLNu4K+w4CUtDQUTiQo3UFB6/aQD109MYNymfXfsPhR0pIWkoiEjcaFY/nQmjB7Bm617uUXFeIDQURCSunNXbROQfAAAHb0lEQVSxCfcO6cbrCzfy9Iefhx0n4WgoiEjc+dYFHbmyV3N+8fpS8larOK86aSiISNwxMx4e3o82jWoz/rkCFedVIw0FEYlLGelpPDlmINv3HuL251WcV100FEQkbvVslcFPr+3Nvz7bwiNvLAs7TkLQUBCRuDY8py2jBrXliXc+460lm8KOE/c0FEQk7v33Nb3o1SqDu1ScV2UaCiIS99LTUpk4ZiAA4ybns/+QivMqS0NBRBJCuyZ1eGREfxau28mPVZxXaYEOBTMbYmbLzGylmd1Xzuu1zGxK5PVZZpYVZB4RSWyX9WzOuIs68fyna5mWr+K8yghsKJhZKvAEcBXQExhtZj1PWOybwDZ37ww8CvwyqDwikhy+e3lXzunYhO+/soClG3eGHSfuBLmmMBhY6e6r3P0g8AIw7IRlhgHPRu5PBS41Mwswk4gkuBqpKTw2egAZ6WmMm1TAThXnnZEaAf7s1kDhcY+LgLNOtoy7HzazHUATYHOAuUQkwWXWr8UTY7IZ9dQnXP7Ie2Skp4UdqVqMHNSW/7igY6C/I8ihUN43/hNPOazIMpjZWGAsQLt27aqeTEQS3qCsxjw+egAz5q8PO0q1aVqvVuC/I8ihUAS0Pe5xG+DEf51jyxSZWQ2gAfCFdit3fwp4CiAnJ0fnsotIhQzt05KhfVqGHSOuBLlPYTbQxcw6mFlNYBQw/YRlpgNfi9y/EXjbVZAuIhKawNYUIvsIbgP+AaQCz7j7IjN7EMhz9+nA08BfzWwlpWsIo4LKIyIipxfk5iPcfSYw84Tnfnjc/f3A8CAziIhIxemMZhERKaOhICIiZTQURESkjIaCiIiU0VAQEZEyFm+nBZhZCbCmkn+9KYlToaH3EpsS5b0kyvsAvZdj2rt75ukWiruhUBVmlufuOWHnqA56L7EpUd5LorwP0Hs5U9p8JCIiZTQURESkTLINhafCDlCN9F5iU6K8l0R5H6D3ckaSap+CiIicWrKtKYiIyCkkzVAwsyFmtszMVprZfWHnqSwze8bMis1sYdhZqsLM2prZO2a2xMwWmdkdYWeqLDNLN7NPzWxe5L38OOxMVWVmqWY2x8xmhJ2lKsxstZktMLO5ZpYXdp7KMrOGZjbVzJZG/s+cE9jvSobNR2aWCiwHLqf0wj6zgdHuvjjUYJVgZhcCu4G/uHvvsPNUlpm1BFq6e4GZ1QfygWvj9N/EgLruvtvM0oAPgTvc/ZOQo1Wamd0N5AAZ7n512Hkqy8xWAznuHtfnKZjZs8AH7v7HyPVp6rj79iB+V7KsKQwGVrr7Knc/CLwADAs5U6W4+/uUc3W6eOPuG9y9IHJ/F7CE0mt2xx0vtTvyMC1yi9tvW2bWBvgy8MewswiYWQZwIaXXn8HdDwY1ECB5hkJroPC4x0XE6QdQIjKzLGAAMCvcJJUX2dwyFygG3nD3uH0vwG+Be4CjYQepBg7808zyI9d6j0cdgRLgT5FNen80s7pB/bJkGQpWznNx+00ukZhZPWAacKe77ww7T2W5+xF370/ptcgHm1lcbtozs6uBYnfPDztLNTnP3bOBq4Dxkc2v8aYGkA1MdPcBwB4gsP2iyTIUioC2xz1uA6wPKYtERLa/TwMmu/tLYeepDpHV+neBISFHqazzgGsi2+JfAC4xs0nhRqo8d18f+bMYeJnSTcnxpggoOm7tcyqlQyIQyTIUZgNdzKxDZCfNKGB6yJmSWmTn7NPAEnd/JOw8VWFmmWbWMHK/NnAZsDTcVJXj7ve7ext3z6L0/8nb7n5zyLEqxczqRg5iILK55Qog7o7ac/eNQKGZdYs8dSkQ2AEZgV6jOVa4+2Ezuw34B5AKPOPui0KOVSlm9jxwEdDUzIqAH7n70+GmqpTzgK8CCyLb4gEeiFzXO960BJ6NHOWWAuS6e1wfypkgmgMvl37/oAbwnLv/PdxIlfYdYHLkS+0q4BtB/aKkOCRVREQqJlk2H4mISAVoKIiISBkNBRERKaOhICIiZTQURESkjIaCyAnM7EikVfPYrdrOHjWzrHhvuJXElhTnKYicoX2RygqRpKM1BZEKinTz/zJy7YRPzaxz5Pn2ZvaWmc2P/Nku8nxzM3s5cp2FeWZ2buRHpZrZHyLXXvhn5CxokZigoSDyRbVP2Hw08rjXdrr7YGACpW2iRO7/xd37ApOBxyLPPwa85+79KO2qOXYWfRfgCXfvBWwHbgj4/YhUmM5oFjmBme1293rlPL8auMTdV0XK/Da6exMz20zpBYMORZ7f4O5NzawEaOPuB477GVmUVmt3iTy+F0hz958G/85ETk9rCiJnxk9y/2TLlOfAcfePoH17EkM0FETOzMjj/vw4cv9flDaKAoyh9HKcAG8B46DsIjwZ0QopUln6hiLyRbWPa24F+Lu7HzsstZaZzaL0C9XoyHO3A8+Y2fcovULWsQbLO4CnzOyblK4RjAM2BJ5epAq0T0GkghLlIvAip6LNRyIiUkZrCiIiUkZrCiIiUkZDQUREymgoiIhIGQ0FEREpo6EgIiJlNBRERKTM/wLHlBLCnWVK7AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "X = [[1,0,0],\n",
    "    [1,1,0],\n",
    "    [1,0,1],\n",
    "    [1,1,1]]\n",
    "\n",
    "y = [0, 0, 0, 1]\n",
    "\n",
    "w = perceptron_train(np.array(X),y)\n",
    "print('Learned Weights:', w)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Predict AND with The learned weights of the Perceptron"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1, 0, 0] 0  AND  0  =  0\n",
      "[1, 1, 0] 1  AND  0  =  0\n",
      "[1, 0, 1] 0  AND  1  =  0\n",
      "[1, 1, 1] 1  AND  1  =  1\n"
     ]
    }
   ],
   "source": [
    "for i in range(4):\n",
    "    print(X[i], X[i][1],' AND ', X[i][2],' = ', predict(X[i],w))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}