my_NN.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "bc37e5ec-92f5-4e40-9355-19f2fe12003e",
   "metadata": {},
   "source": [
    "In this notebook I would like to implement my first **Neural Network** from scratch.\n",
    "To do so, I will apply it to the [MNIST database](https://en.wikipedia.org/wiki/MNIST_database): a large database of handwritten digits. We will predict if a given digit is a zero or not.\n",
    "\n",
    "The main idea of [neural networks](https://en.wikipedia.org/wiki/Artificial_neural_network) is the following.\n",
    "- We start with some random parameters W and b.\n",
    "- Next, we implement the **forward propagation step**. This has several *layers*, each of which predicts an output\n",
    "$ g (W * A + b) $,\n",
    "where\n",
    "$ A $ \n",
    "is the output of the previous layer and \n",
    "$ g $ \n",
    "is the [*activation function*](https://en.wikipedia.org/wiki/Activation_function). \n",
    "Clearly, the input of the fist layer is an image of the dataset.\n",
    "- We compute the *Cross-Entropy cost* of the final output: this tends to zero when we make correct predictions.\n",
    "- At this point, our goal is to minimize the cost to make better predictions using **gradient descent**. To do so, we look for a minimum of the cost function by taking the derivatives of the cost function and \"making a step towards the minimum\" changing the parameters W and b. This is called the **backward propagation step**.\n",
    "\n",
    "Now, first of all let's import the libraries."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "7b8e2007-76de-4560-930a-8f45b2bc7d23",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np \n",
    "\n",
    "%matplotlib inline\n",
    "import matplotlib as mpl \n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "from sklearn.datasets import fetch_openml"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c285d434-fefa-4ade-ac59-2346860c28d7",
   "metadata": {},
   "source": [
    "Here are some **notations for parameters and hyperparameters**:\n",
    "\n",
    "- *L* will denote the number of layers, for example L=3\n",
    "- *layer_dimension* has all the information about number of nodes in each layer: it is a list of length L\n",
    "- the learning rate for the gradient descent is *alpha* (this is the length of the step we're making)\n",
    "- *W* and *b* are the parameters for the linear steps, to be initialised at random\n",
    "- *activs* has the information about the activation function of the hidden and output layers: it is a list [hidden activation, output activation]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "3f9f4f55-8578-4bb8-bb53-e8b87795821c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# random initialization of W and b\n",
    "\n",
    "def initialization(layer_dimensions):\n",
    "    param = {}\n",
    "    L = len(layer_dimensions)\n",
    "    for l in range(1, L):\n",
    "        param['W'+str(l)] = np.random.randn(layer_dimensions[l], layer_dimensions[l-1])*np.sqrt(2. / layer_dimensions[l-1])\n",
    "        param['b'+str(l)] = np.zeros((layer_dimensions[l],1))\n",
    "    return param"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "90637747-ec92-4540-bf2c-fcbd4be1934a",
   "metadata": {},
   "source": [
    "**Forward propagation** steps:\n",
    "\n",
    "1) compute the output of the $ l $-th layer $ Z[l] = W[l] * A[l-1] + b[l] $\n",
    "\n",
    "2) apply the activation function $ g $ to $ Z[l] $ and put $ A[l] = g(Z[l]) $\n",
    "\n",
    "3) store the values in cache for the backpropagation\n",
    "\n",
    "4) iterate for every layer $ l= 1, \\dots , L $"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "4a4d015b-a197-43c3-8542-41d0f370aca5",
   "metadata": {},
   "outputs": [],
   "source": [
    "# the next function applies the activation g to the linear combination of W, A and b\n",
    "def activ_forward(A_prev, W, b, activ):\n",
    "    \n",
    "    Z = np.dot(W,A_prev) + b\n",
    "    lin_cache = (A_prev, W, b)\n",
    "    \n",
    "    if activ == 'tanh':\n",
    "        A = np.tanh(Z)\n",
    "        \n",
    "    elif activ == 'relu':\n",
    "        A = np.maximum(Z,0)\n",
    "        \n",
    "    elif activ == 'sigmoid':\n",
    "        A = 1. /(1. + np.exp(-Z))\n",
    "        \n",
    "    else:\n",
    "        A = None\n",
    "        print('Unknown activation function!')\n",
    "    \n",
    "    cache = (lin_cache, Z)\n",
    "    return A, cache\n",
    "\n",
    "# the following function implements forward propagation as explained above\n",
    "def model_forward(X, param, activs= ['relu', 'sigmoid']):\n",
    "    \n",
    "    caches = []\n",
    "    A = X\n",
    "    L = len(param) // 2\n",
    "    \n",
    "    # implement linear transformation and activs[0] for the first L-1 layers\n",
    "    for l in range(1, L):\n",
    "        A_prev = A \n",
    "        A, cache = activ_forward(A_prev, param['W'+str(l)], param['b'+str(l)], activs[0])\n",
    "        caches.append(cache)\n",
    "    \n",
    "    # in the ouput layer, use activs[1]\n",
    "    AL, cache = activ_forward(A, param['W'+str(L)], param['b'+str(L)], activs[1])\n",
    "    caches.append(cache)\n",
    "    \n",
    "    return AL, caches"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "895e4fa6-50e6-4a68-b50e-babc69be0cde",
   "metadata": {},
   "source": [
    "Next, we **compute the cost**. Recall that the *Cross-Entropy cost* is \n",
    "\n",
    "$ J = - \\frac{1}{m} \\sum_{i=1}^m y^i \\log (A^{[L]i}) + (1-y^i) \\log (1-A^{[L]i}) $\n",
    "\n",
    "where m is the number of training examples."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "73a336da-fc5d-47e0-a637-78c756a4a7d8",
   "metadata": {},
   "outputs": [],
   "source": [
    "def comp_cost(AL, Y):\n",
    "     \n",
    "    m = Y.shape[1]\n",
    "    cost = -np.sum((np.multiply(np.log(AL),Y) + np.multiply(np.log(1-AL),1-Y))/m)\n",
    "    return np.squeeze(cost) # To make sure your cost's shape is what we expect (e.g. this turns [[7]] into 7)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "85c67e5e-ecb5-4a6c-a203-9e0e8fc170a3",
   "metadata": {},
   "source": [
    "We now turn to the **backward propagation**. To do so, we have to \n",
    "\n",
    "1) compute $ dA[L] $, the derivative of the cost $ J $ with respect to $ A[L] $\n",
    "\n",
    "2) calculate the gradients $ dZ[l] = dA[l] * g'(Z[l]) $, where $ g' $ is the derivative of the activation function $ g $\n",
    "\n",
    "3) compute $ dW[l], db[l], dA[l-1] $\n",
    "\n",
    "4) iterate over $ l = L, \\dots, 1 $"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "900ecf81-844f-4152-9079-9494114f2540",
   "metadata": {},
   "outputs": [],
   "source": [
    "# compute g'(Z):\n",
    "def der_activ(Z, activ):\n",
    "    \n",
    "    if activ == 'relu':\n",
    "        derivate = Z\n",
    "        derivate[Z<=0] = 0\n",
    "        derivate[Z>0] = 1\n",
    "        \n",
    "    elif activ == 'tanh':\n",
    "        a = np.tanh(Z)\n",
    "        derivate = 1 - a**2\n",
    "        \n",
    "    elif activ == 'sigmoid':\n",
    "        a = 1/(1+np.exp(-Z))\n",
    "        derivate = a*(1-a) \n",
    "        \n",
    "    else:\n",
    "        derivate = None\n",
    "        print('Unknown activation function!')\n",
    "    \n",
    "    return derivate\n",
    "\n",
    "# steps 2) and 3):\n",
    "def activ_backward(dA_prev, cache, activ):\n",
    "    \n",
    "    A_prev, W, b = cache[0]\n",
    "    Z = cache[1]\n",
    "    \n",
    "    dZ = np.multiply(dA_prev, der_activ(Z, activ))\n",
    "    \n",
    "    m = A_prev.shape[1]\n",
    "    dW = np.dot(dZ, A_prev.T) / m\n",
    "    db = np.sum(dZ, axis=1, keepdims= True) / m\n",
    "    dA = np.dot(W.T, dZ)\n",
    "    \n",
    "    return dA, dW, db\n",
    "\n",
    "# the iteration:\n",
    "def model_backward(AL, Y, caches, activs = ['relu', 'sigmoid']):\n",
    "    \n",
    "    grads = {}\n",
    "    L = len(caches) # the number of layers\n",
    "    m = AL.shape[1]\n",
    "    Y = Y.reshape(AL.shape) \n",
    "    \n",
    "    # fist step of backprop:\n",
    "    dAL = - (np.divide(Y, AL) - np.divide(1 - Y, 1 - AL))\n",
    "    \n",
    "    current_cache = caches[L-1]\n",
    "    dA_prev, dW_temp, db_temp = activ_backward(dAL, current_cache, activs[1])\n",
    "    grads['dA' + str(L-1)] = dA_prev\n",
    "    grads['dW' + str(L)] = dW_temp\n",
    "    grads['db' + str(L)] = db_temp\n",
    "    \n",
    "    # loop from l=L-2 to l=0\n",
    "    for l in reversed(range(L-1)):\n",
    "        current_cache = caches[l]\n",
    "        dA_prev, dW_temp, db_temp = activ_backward(grads['dA'+str(l+1)], current_cache, activs[0])\n",
    "        grads['dA' + str(l)] = dA_prev\n",
    "        grads['dW' + str(l+1)] = dW_temp\n",
    "        grads['db' + str(l+1)] = db_temp\n",
    "\n",
    "    return grads"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "00653b59-ece5-499b-b248-1ed27fc6ac03",
   "metadata": {},
   "source": [
    "Now that we have all the gradients, we have to update the parameters W and b following the rule\n",
    "\n",
    "$ X[l] = W[l] - \\alpha * dW[l] $\n",
    "\n",
    "$ b[l] = b[l] - \\alpha * db[l] $\n",
    "\n",
    "where $ \\alpha $ is the learning rate."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "40f54ba9-ea21-4165-ae74-18d87b1d1fab",
   "metadata": {},
   "outputs": [],
   "source": [
    "def update_parameters(params, grads, alpha):\n",
    "    \n",
    "    param = params.copy()\n",
    "    L = len(param) // 2 \n",
    "\n",
    "    for l in range(L):\n",
    "        param['W' + str(l+1)] = param['W' + str(l+1)] - alpha * grads['dW'+str(l+1)]\n",
    "        param['b' + str(l+1)] = param['b' + str(l+1)] - alpha * grads['db'+str(l+1)]\n",
    "        \n",
    "    return param"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5acd3d5d-a2cc-4b74-8941-746069e83647",
   "metadata": {},
   "source": [
    "#### We finally have all the functions we need to **implement our model**!!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "82043b3f-a29f-4ac7-81d3-6810adda8f99",
   "metadata": {},
   "outputs": [],
   "source": [
    "def model(X, Y, layer_dimensions, alpha = 0.0075, num_iterations = 3000, activs = ['relu','sigmoid']):\n",
    "    \n",
    "    costs = []\n",
    "    param = initialization(layer_dimensions)\n",
    "    \n",
    "    # implement gradient descent\n",
    "    for i in range(0, num_iterations):\n",
    "\n",
    "        # forward propagation\n",
    "        AL, caches = model_forward(X, param, activs)\n",
    "        \n",
    "        # compute cost\n",
    "        cost = comp_cost(AL, Y)\n",
    "        \n",
    "        # backward propagation\n",
    "        grads = model_backward(AL, Y, caches, activs)\n",
    "        \n",
    "        # update parameters\n",
    "        param = update_parameters(param, grads, alpha)\n",
    "                \n",
    "        # save the cost every 100 iterations\n",
    "        if i % 100 == 0 or i == num_iterations:\n",
    "            costs.append(cost)\n",
    "    \n",
    "    return param, costs"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "976e9144-1bcd-47af-a564-e932166fcda0",
   "metadata": {},
   "source": [
    "Before applying our function to the dataset, we write the function\n",
    "**plot_cost** to have a visualization of the cost during the iterations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "84a03cd4-2e59-4974-9138-091898d488ee",
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_cost(costs, n_iter):\n",
    "    plt.plot(list(range(int(n_iter/100))), costs, '-r')\n",
    "    plt.title('Cost function')\n",
    "    plt.xlabel('iterations (k)')\n",
    "    plt.ylabel('cost')\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "36ab1dbb-8170-4366-badc-f70223e78e30",
   "metadata": {},
   "source": [
    "### Let's apply this to the MNIST dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "cd4a0be0-2110-478b-8532-ec5318854bcf",
   "metadata": {},
   "outputs": [],
   "source": [
    "# import the MNIST dataset:\n",
    "mnist = fetch_openml('mnist_784', version = 1)\n",
    "\n",
    "X, y = mnist['data'], mnist['target']\n",
    "y = y.astype(np.uint8)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "fe79fc8f-2f3d-4fbc-ae13-d912c5a4b71f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Lets plot some digits!\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAOcAAADnCAYAAADl9EEgAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAGaElEQVR4nO3dPUiWfR/G8dveSyprs2gOXHqhcAh6hZqsNRqiJoPKRYnAoTGorWyLpqhFcmgpEmqIIByKXiAHIaKhFrGghiJ81ucBr991Z/Z4XPr5jB6cXSfVtxP6c2rb9PT0P0CeJfN9A8DMxAmhxAmhxAmhxAmhljXZ/Vcu/H1tM33RkxNCiRNCiRNCiRNCiRNCiRNCiRNCiRNCiRNCiRNCiRNCiRNCiRNCiRNCiRNCiRNCiRNCiRNCiRNCiRNCiRNCiRNCiRNCiRNCiRNCiRNCiRNCiRNCiRNCiRNCiRNCiRNCiRNCiRNCiRNCiRNCiRNCiRNCLZvvG+B//fr1q9y/fPnyVz9/aGio4fb9+/fy2vHx8XK/ceNGuQ8MDDTc7t69W167atWqcr948WK5X7p0qdzngycnhBInhBInhBInhBInhBInhBInhHLOOYMPHz6U+48fP8r92bNn5f706dOG29TUVHnt8PBwuc+nLVu2lPv58+fLfWRkpOG2du3a8tpt27aV+759+8o9kScnhBInhBInhBInhBInhBInhGqbnp6u9nJsVS9evCj3gwcPlvvffm0r1dKlS8v91q1b5d7e3j7rz960aVO5b9iwody3bt0668/+P2ib6YuenBBKnBBKnBBKnBBKnBBKnBBKnBBqUZ5zTk5Olnt3d3e5T0xMzOXtzKlm997sPPDx48cNtxUrVpTXLtbz3zngnBNaiTghlDghlDghlDghlDghlDgh1KL81pgbN24s96tXr5b7/fv3y33Hjh3l3tfXV+6V7du3l/vo6Gi5N3un8s2bNw23a9euldcytzw5IZQ4IZQ4IZQ4IZQ4IZQ4IZQ4IdSifJ/zT339+rXcm/24ut7e3obbzZs3y2tv375d7idOnCh3InmfE1qJOCGUOCGUOCGUOCGUOCGUOCHUonyf80+tW7fuj65fv379rK9tdg56/Pjxcl+yxL/HrcKfFIQSJ4QSJ4QSJ4QSJ4QSJ4Tyytg8+PbtW8Otp6envPbJkyfl/uDBg3I/fPhwuTMvvDIGrUScEEqcEEqcEEqcEEqcEEqcEMo5Z5iJiYly37lzZ7l3dHSU+4EDB8p9165dDbezZ8+W17a1zXhcR3POOaGViBNCiRNCiRNCiRNCiRNCiRNCOedsMSMjI+V++vTpcm/24wsrly9fLveTJ0+We2dn56w/e4FzzgmtRJwQSpwQSpwQSpwQSpwQSpwQyjnnAvP69ety7+/vL/fR0dFZf/aZM2fKfXBwsNw3b948689ucc45oZWIE0KJE0KJE0KJE0KJE0KJE0I551xkpqamyv3+/fsNt1OnTpXXNvm79M+hQ4fK/dGjR+W+gDnnhFYiTgglTgglTgglTgglTgjlKIV/beXKleX+8+fPcl++fHm5P3z4sOG2f//+8toW5ygFWok4IZQ4IZQ4IZQ4IZQ4IZQ4IdSy+b4B5tarV6/KfXh4uNzHxsYabs3OMZvp6uoq97179/7Rr7/QeHJCKHFCKHFCKHFCKHFCKHFCKHFCKOecYcbHx8v9+vXr5X7v3r1y//Tp02/f07+1bFn916mzs7PclyzxrPhvfjcglDghlDghlDghlDghlDghlDghlHPOv6DZWeKdO3cabkNDQ+W179+/n80tzYndu3eX++DgYLkfPXp0Lm9nwfPkhFDihFDihFDihFDihFDihFCOUmbw+fPncn/79m25nzt3rtzfvXv32/c0V7q7u8v9woULDbdjx46V13rla2753YRQ4oRQ4oRQ4oRQ4oRQ4oRQ4oRQC/acc3JysuHW29tbXvvy5ctyn5iYmM0tzYk9e/aUe39/f7kfOXKk3FevXv3b98Tf4ckJocQJocQJocQJocQJocQJocQJoWLPOZ8/f17uV65cKfexsbGG28ePH2d1T3NlzZo1Dbe+vr7y2mbffrK9vX1W90QeT04IJU4IJU4IJU4IJU4IJU4IJU4IFXvOOTIy8kf7n+jq6ir3np6ecl+6dGm5DwwMNNw6OjrKa1k8PDkhlDghlDghlDghlDghlDghlDghVNv09HS1lyMwJ9pm+qInJ4QSJ4QSJ4QSJ4QSJ4QSJ4QSJ4QSJ4QSJ4QSJ4QSJ4QSJ4QSJ4QSJ4QSJ4QSJ4QSJ4QSJ4QSJ4QSJ4QSJ4QSJ4Rq9iMAZ/yWfcDf58kJocQJocQJocQJocQJocQJof4DO14Dh4wBfawAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAOcAAADnCAYAAADl9EEgAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAGrUlEQVR4nO3dX2jPexzH8e90kqIt+VNTcuWeceVmw40kLtBcrJSUKBRyIRcLF3KhFBcuTflTEjXXuKKVNbnb7RQXUlsiUjvXp/Z7/zqbP69tj8elV1/7NufZt86n3/fXMT093QB5lvztGwBmJk4IJU4IJU4IJU4I9U+b3f/Khd+vY6Y/9OSEUOKEUOKEUOKEUOKEUOKEUOKEUOKEUOKEUOKEUOKEUOKEUOKEUOKEUOKEUOKEUOKEUOKEUOKEUOKEUOKEUOKEUOKEUOKEUOKEUOKEUOKEUOKEUOKEUOKEUO2+ApAF5s2bN+V+8+bNltudO3fKaw8fPlzuJ0+eLPeenp5yX2w8OSGUOCGUOCGUOCGUOCGUOCGUOCFUx/T0dLWXI3nGxsbKffv27eU+NTX1C+/mv7q6usr98+fPv+1nh+uY6Q89OSGUOCGUOCGUOCGUOCGUOCGUOCGUz3POMyMjI+W+f//+cp+cnCz3jo4Zj9yapmmazs7O8tqlS5eW+6dPn8r91atXLbctW7bM6WfPR56cEEqcEEqcEEqcEEqcEEqcEMpHxv6Cr1+/ttxGR0fLawcGBsp9YmKi3Nv8e5dHKe2OM86fP1/u/f395V7d25UrV8prL1y4UO7hfGQM5hNxQihxQihxQihxQihxQihxQigfGfsLjh071nK7d+/eH7yT/6fd1wd++fKl3Ht7e8v9xYsXLbd3796V1y5EnpwQSpwQSpwQSpwQSpwQSpwQSpwQyjnnb9DuPHB4eLjl1u7zlu309fWV+549e8r93LlzLbd169aV127evLncV65cWe7Pnz9vuc319zIfeXJCKHFCKHFCKHFCKHFCKHFCKHFCKO+tnYWxsbFy3759e7lPTU3N+mfv3r273O/fv1/u1Wcmm6b+3OTRo0fLa9esWVPu7SxZ0vpZsXz58vLaly9flntPT8+s7ukP8d5amE/ECaHECaHECaHECaHECaHECaGcc85gfHy83AcHB8v9wYMH5V6dB3Z3d5fXXrx4sdwPHDhQ7smqc87qe0Obpv13fya/D7hxzgnzizghlDghlDghlDghlDgh1KJ8Neb379/LvXo9ZNM0zbNnz8q9s7Oz3IeGhlpuW7duLa/99u1buS9WExMTf/sWfjlPTgglTgglTgglTgglTgglTgglTgi1KM85R0dHy73dOWY7T58+Lffe3t45/f0sDp6cEEqcEEqcEEqcEEqcEEqcEEqcEGpRnnOeOXOm3Nu8LrTp6+srd+eYs9Pu9/67rk3lyQmhxAmhxAmhxAmhxAmhxAmhxAmhFuw55/DwcMttbGysvLbd183t3bt3NrdEG9Xvvd2/yaZNm37x3fx9npwQSpwQSpwQSpwQSpwQSpwQSpwQasGec1bfY/njx4/y2rVr15Z7f3//rO5poWv3vaeDg4Oz/rt37txZ7levXp31353KkxNCiRNCiRNCiRNCiRNCiRNCLdijlLlYtmxZuXd3d/+hO8nS7qjkypUr5X7t2rVyX79+fcvt7Nmz5bUrVqwo9/nIkxNCiRNCiRNCiRNCiRNCiRNCiRNCOeecwWJ+9WX12tB255QPHz4s93379pX748ePy32x8eSEUOKEUOKEUOKEUOKEUOKEUOKEUAv2nHN6enpWW9M0zZMnT8r9xo0bs7mlCNevXy/3y5cvt9wmJyfLawcGBsp9aGio3PkvT04IJU4IJU4IJU4IJU4IJU4IJU4ItWDPOTs6Oma1NU3TfPz4sdxPnTpV7keOHCn3VatWtdxev35dXnv37t1yf/v2bblPTEyU+4YNG1puu3btKq89ceJEufP/eHJCKHFCKHFCKHFCKHFCKHFCqAV7lDIXP3/+LPdbt26V+6NHj8q9q6ur5TY+Pl5eO1fbtm0r9x07drTcLl269Ktvh4InJ4QSJ4QSJ4QSJ4QSJ4QSJ4QSJ4TqaPOayPodksHev3/fcjt48GB57cjIyJx+drtXb7b7yFpl9erV5X7o0KFyn8+v9VzAZvwPwpMTQokTQokTQokTQokTQokTQokTQi3Yc87Khw8fyv327dvlXn1NXtPM7Zzz9OnT5bXHjx8v940bN5Y7kZxzwnwiTgglTgglTgglTgglTgglTgi1KM85IYxzTphPxAmhxAmhxAmhxAmhxAmhxAmhxAmhxAmhxAmhxAmhxAmhxAmhxAmhxAmhxAmhxAmhxAmhxAmhxAmhxAmhxAmhxAmhxAmhxAmhxAmhxAmhxAmhxAmhxAmh/mmzz/jVZMDv58kJocQJocQJocQJocQJocQJof4Ftv8iCGE1mZwAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAOcAAADnCAYAAADl9EEgAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAFa0lEQVR4nO3dsUvUfxzH8e5HIQVCW0NiEMiVjZ1oTS0NTim0heDQ1By09HdE3BbU0h8gIhxUQw5NtckNhQRmUIlLUHDX1PLD+5xd131fp4/H6JuPvRGefKAPp7Vut3sCyPNf1QsABxMnhBInhBInhBInhDrZZ+6/cuHfqx30RTcnhBInhBInhBInhBInhBInhBInhBInhBInhBInhBInhBInhBInhBInhBInhBInhBInhBInhBInhBInhBInhBInhBInhBInhBInhBInhBInhBInhBInhBInhBInhBInhBInhBInhBInhBInhBInhBInhDpZ9QLwW6vV6jm7c+dO8ezLly+L83q9PtBOVXJzQihxQihxQihxQihxQihxQihxQqjYd85Xr14V51++fCnOl5eXh7kOI/DmzZues0ajMcJNMrg5IZQ4IZQ4IZQ4IZQ4IZQ4IVTsU8qLFy+K83a7XZx7SsnT6XSK8/fv3/ecbW9vF892u92Bdkrm5oRQ4oRQ4oRQ4oRQ4oRQ4oRQ4oRQse+cT548Kc6vX78+ok0Ylp2dneK82Wz2nK2srBTPXrp0aaCdkrk5IZQ4IZQ4IZQ4IZQ4IZQ4IZQ4IVTsO2e/z/4xfu7evTvw2ZmZmSFuMh7cnBBKnBBKnBBKnBBKnBBKnBBKnBCqsnfOd+/eFee7u7sj2oRR2dvbG/jszZs3h7fImHBzQihxQihxQihxQihxQihxQihxQqjK3jnX1taK8+/fv49oE4al39v0hw8fBv7e58+fH/jsuHJzQihxQihxQihxQihxQihxQqjKnlK2trb+6vyVK1eGtAnDcv/+/eL806dPxXm9Xu85m5ycHGincebmhFDihFDihFDihFDihFDihFDihFCxfwKwn7m5uapXGEv7+/vF+fr6es/Z06dPi2c3NjYG2um3hw8f9pydPXv2r773OHJzQihxQihxQihxQihxQihxQihxQqixfef8+vVrZf/227dvi/NOp1Oct1qtnrOPHz8Wz/748aM4f/bsWXHeb7fTp0/3nM3PzxfPTkxMFOc/f/4szhuNRnF+3Lg5IZQ4IZQ4IZQ4IZQ4IZQ4IZQ4IVSt2+2W5sXh37h3715x/vjx4+K83+f7Lly48KcrHVq/d84+P9MTp06d6jk7c+ZM8ezly5eL84WFheL86tWrxfmNGzd6zs6dO1c8OzU1VZx/+/atOO/3hnuE1Q76opsTQokTQokTQokTQokTQokTQokTQlX2ec5Hjx4V5/3eKV+/fj3Mdf7I9PR0cX7r1q3ifHZ2tues3ztllZrNZnH++fPn4vzixYvDXOfIc3NCKHFCKHFCKHFCKHFCKHFCqNhfjfngwYOqV+B/Sr/S8zBu3749pE2OBzcnhBInhBInhBInhBInhBInhBInhIp95+ToWVpaqnqFseLmhFDihFDihFDihFDihFDihFDihFDihFDihFDihFDihFDihFDihFDihFDihFA+z8nItNvt4vzatWsj2mQ8uDkhlDghlDghlDghlDghlDghlKcURqbT6VS9wlhxc0IocUIocUIocUIocUIocUIocUIo75yMzObmZnG+uro6mkXGhJsTQokTQokTQokTQokTQokTQokTQokTQokTQokTQokTQokTQokTQokTQokTQvk8J4e2uLhYnD9//nxEmxwPbk4IJU4IJU4IJU4IJU4IJU4IJU4IVet2u6V5cQgMRe2gL7o5IZQ4IZQ4IZQ4IZQ4IZQ4IZQ4IZQ4IZQ4IZQ4IZQ4IZQ4IZQ4IZQ4IZQ4IZQ4IZQ4IZQ4IZQ4IZQ4IZQ4IZQ4IVS/PwF44K/sA/49NyeEEieEEieEEieEEieEEieE+gUraqrmUaqlFQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAOcAAADnCAYAAADl9EEgAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAFCklEQVR4nO3dsWpUaQCG4czOgo2FVQI2go25AyGNor2FFyCxVcRKsLKy0MLGK9BSKwuLiBaCXoDgDWgjCAFFsLAIs93Cwsyf3SSzvkmep8zHnDPNyw85zMxkNputAD1//O43AMwnTogSJ0SJE6LECVF/7rL7Vy4s32TeH52cECVOiBInRIkTosQJUeKEKHFClDghSpwQJU6IEidEiROixAlR4oQocUKUOCFKnBAlTogSJ0SJE6LECVHihChxQpQ4IUqcECVOiBInRIkTosQJUeKEKHFClDghSpwQJU6IEidEiROixAlR4oSoP3/3G+DouH///nC/d+/ecJ/NZgu3t2/fDl974cKF4X4YOTkhSpwQJU6IEidEiROixAlR4oQozzn51548eTLcHzx4MNyn0+lw39nZWbhNJpPha48iJydEiROixAlR4oQocUKUOCHKoxT+tc+fPw/3X79+/U/v5HhwckKUOCFKnBAlTogSJ0SJE6LECVGec/IPb968Wbg9fvx4X9deX18f7i9fvly4ra2t7eveh5GTE6LECVHihChxQpQ4IUqcECVOiPKc85h5//79cN/c3Fy4/fjxY1/3vnPnznA/c+bMvq5/1Dg5IUqcECVOiBInRIkTosQJUeKEKM85j5mnT58O9y9fvuz52hcvXhzu165d2/O1jyMnJ0SJE6LECVHihChxQpQ4IUqcEDWZzWajfTjSs729PdxXV1eH+3Q6XbidOnVq+Npnz54N90uXLg33Y2wy749OTogSJ0SJE6LECVHihChxQpSPjB0ynz59Gu5Xr15d2r1v3bo13D0qOVhOTogSJ0SJE6LECVHihChxQpQ4IcpzzkNma2truH/8+HFf1798+fLC7fbt2/u6Nv+NkxOixAlR4oQocUKUOCFKnBAlTojy1ZgxL168GO6bm5vD/efPn8N9Y2NjuD9//nzhtra2Nnwte+arMeEwESdEiROixAlR4oQocUKUOCHK5zl/g9F3zy7ze2dXVlZWzp49O9w9y+xwckKUOCFKnBAlTogSJ0SJE6LECVGec/4GDx8+XLhNp9Ol3vvu3btLvT4Hx8kJUeKEKHFClDghSpwQJU6I8ihlCT58+DDcX716tbR7X7lyZbifO3duaffmYDk5IUqcECVOiBInRIkTosQJUeKEKD8BuASrq6vD/du3b3u+9vnz54f71tbWcD958uSe783S+AlAOEzECVHihChxQpQ4IUqcECVOiPJ5ziXY3t4e7vv5+subN28Od88xjw4nJ0SJE6LECVHihChxQpQ4IUqcEOU55x5cv359uO/yGdmVnZ2dPd97Y2Njz6/lcHFyQpQ4IUqcECVOiBInRIkTojxKmWO3n/B7/fr1cJ9M5n7T4d9OnDixcLtx48bwtWtra8Odo8PJCVHihChxQpQ4IUqcECVOiBInRHnOOcf379+H+9evX/d1/dOnTy/cHj16tK9rc3Q4OSFKnBAlTogSJ0SJE6LECVHihChxQpQ4IUqcECVOiBInRIkTosQJUeKEKJ/nnGN9fX247/YzfO/evTvIt8Mx5eSEKHFClDghSpwQJU6IEidEiROiJrPZbLQPR+BAzP1BVycnRIkTosQJUeKEKHFClDghSpwQJU6IEidEiROixAlR4oQocUKUOCFKnBAlTogSJ0SJE6LECVHihChxQpQ4IWq3nwCc+5V9wPI5OSFKnBAlTogSJ0SJE6LECVF/Aepbi9y1gU/6AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAOcAAADnCAYAAADl9EEgAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAGLUlEQVR4nO3dwYtNfRzH8TuTQlOSopCNKJSdmg2NFRuyoElJ1GRl4Q/ARlIWdrI0igUbsRxWxkrsmI1INshQIqHUPKvnqafmfE/uzczn3nm9lvPp3Lml95zy69w7NDc31wHyDC/2GwDmJ04IJU4IJU4IJU4Itaxl91+58PcNzfdDd04IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4ItWyx3wD/9+TJk3K/efNmuU9PT5f7ixcv/vg9/evKlSvlvmHDhnJ//PhxuR8/frxxGx0dLa8dRO6cEEqcEEqcEEqcEEqcEEqcEEqcEGpobm6u2suR7ty5c6dxO3PmTHnt7Oxsubf8e3b27t1b7p8+fWrcZmZmymvbtL238fHxxu327ds9/e5wQ/P90J0TQokTQokTQokTQokTQokTQokTQnmeswu/f/8u96dPn5b7qVOnGrfv37+X146NjZX7+fPny3337t3l/uvXr8atOofsdDqdqampcm+za9eunq4fNO6cEEqcEEqcEEqcEEqcEEqcEEqcEMo5Zxdu3bpV7hMTE12/9r59+8q9eha00+l0Vq1a1fXvbnv9Xs8xN23aVO4nTpzo6fUHjTsnhBInhBInhBInhBInhBInhBInhPK5tfM4d+5cuV+6dKnch4bm/RjS/5w+fbpxu3jxYnltr+eYbbZv3964vXz5sqfXvnv3brkfOnSop9fvYz63FvqJOCGUOCGUOCGUOCGUOCHUknxk7MKFC+XedlSyfPnyct+/f3+5X758uXFbuXJleW2bnz9/lvuDBw/K/e3bt41b21f4tX0s5xI+KumKOyeEEieEEieEEieEEieEEieEEieEGthHxr58+dK4bdu2rbx2dna23A8ePFju9+7dK/devHr1qtyPHTtW7s+ePev6dx85cqTcr1+/Xu4jIyNd/+4B55Ex6CfihFDihFDihFDihFDihFDihFADe8758ePHxm39+vU9vfabN2/KfcWKFeU+OTnZuN2/f7+8dmZmpty/fftW7m0f2zk83Pz3uu2jLdvOf2nknBP6iTghlDghlDghlDghlDghlDgh1MCec1bPc1Zfc9fp1GeknU7757e2nSX2YuPGjeXe9t7evXtX7uvWrWvc3r9/X15L15xzQj8RJ4QSJ4QSJ4QSJ4QSJ4QSJ4Qa2O/nXL16dePW9rmyBw4cKPfPnz+X+5YtW8q9+p7KkydPlteuWbOm3I8ePVrubeecbdezcNw5IZQ4IZQ4IZQ4IZQ4IZQ4IdTAHqVURkdHy73tKwAX0/T0dLk/evSo3NseZ9u8efMfvyf+DndOCCVOCCVOCCVOCCVOCCVOCCVOCLUkzzn72Y8fP8q97RyzbffIWA53TgglTgglTgglTgglTgglTgglTgg1sF8BuFQND9d/b9vOOT98+NC4rV27tqv3RCtfAQj9RJwQSpwQSpwQSpwQSpwQSpwQyvOcfWZqamqx3wILxJ0TQokTQokTQokTQokTQokTQjlK6TOvX79e7LfAAnHnhFDihFDihFDihFDihFDihFDihFDOOfvMnj17yr3lo07pI+6cEEqcEEqcEEqcEEqcEEqcEEqcEMo5Z5/ZuXNnuW/durXc254HrXZfAbiw3DkhlDghlDghlDghlDghlDghlDgh1FDL838eDuwzN27cKPeJiYlyHxsba9yuXr1aXrtjx45yp9HQfD9054RQ4oRQ4oRQ4oRQ4oRQ4oRQ4oRQzjkHzNevX8t9fHy83B8+fNi4HT58uLx2cnKy3EdGRsp9CXPOCf1EnBBKnBBKnBBKnBBKnBDKUcoS03bUcvbs2cbt2rVr5bXPnz8vd4+UNXKUAv1EnBBKnBBKnBBKnBBKnBBKnBDKOScsPuec0E/ECaHECaHECaHECaHECaHECaGWtezznr8Af587J4QSJ4QSJ4QSJ4QSJ4QSJ4T6B30hCMScM9K9AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "print('Lets plot some digits!')\n",
    "for i in range(5):\n",
    "    num = np.random.randn(1, X.shape[0])\n",
    "    digit = X.loc[i].values\n",
    "    plt.imshow(digit.reshape(28,28), cmap = 'binary')\n",
    "    plt.axis('off')\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "7523982f-0569-447c-9862-67a197cbf0cf",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Each image is of size (n_x): 784 \n",
      "\n",
      "Shape of the training and testing vectors: \n",
      "\n",
      "X_train: (60000, 784)\n",
      "y_train: (60000,)\n",
      "X_test:  (10000, 784)\n",
      "y_test:  (10000,)\n",
      "\n",
      "Number of training examples (m_train): 60000\n",
      "Number of testing examples: 10000\n"
     ]
    }
   ],
   "source": [
    "print ('Each image is of size (n_x): ' + str(X.shape[1]),'\\n')\n",
    "\n",
    "X_train, X_test, y_train, y_test = X[:60000], X[60000:], y[:60000], y[60000:]\n",
    "\n",
    "print('Shape of the training and testing vectors: \\n')\n",
    "print('X_train: ' + str(X_train.shape))\n",
    "print('y_train: ' + str(y_train.shape))\n",
    "print('X_test:  '  + str(X_test.shape))\n",
    "print('y_test:  '  + str(y_test.shape))\n",
    "\n",
    "m_train = X_train.shape[0]\n",
    "num_px = X_train.shape[1]\n",
    "m_test = X_test.shape[0]\n",
    "\n",
    "print ('\\nNumber of training examples (m_train): ' + str(m_train))\n",
    "print ('Number of testing examples: ' + str(m_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "95cc11bf-73b7-4464-85a7-e8b7595e94eb",
   "metadata": {},
   "source": [
    "According to the functions I wrote, X should have shape (n_x, m_train): I'll take the transpose to have this. \n",
    "Also, y should be of the shape (1, m_train). \n",
    "Same for the test set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "2c6d07da-d035-4ea1-98d6-82ad6ce76b02",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Updated shape of the training and testing vectors: \n",
      "\n",
      "X_train: (784, 60000)\n",
      "y_train: (1, 60000)\n",
      "X_test:  (784, 10000)\n",
      "y_test:  (1, 10000)\n"
     ]
    }
   ],
   "source": [
    "X_train = X_train.T.to_numpy()\n",
    "y_train = y_train.values.reshape((1,X_train.shape[1]))\n",
    "X_test = X_test.T.to_numpy()\n",
    "y_test = y_test.values.reshape((1,X_test.shape[1]))\n",
    "\n",
    "print('Updated shape of the training and testing vectors: \\n')\n",
    "\n",
    "print('X_train: ' + str(X_train.shape))\n",
    "print('y_train: ' + str(y_train.shape))\n",
    "print('X_test:  '  + str(X_test.shape))\n",
    "print('y_test:  '  + str(y_test.shape))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "73b8bebc-cdf2-44da-b605-2d4ea28b69c0",
   "metadata": {},
   "source": [
    "Next, we *normalize* the data to have feature values between 0 and 1. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "87508131-1fd8-4c70-bd24-16df941404bb",
   "metadata": {},
   "outputs": [],
   "source": [
    "X_mean = np.sum(X_train, axis=1)/X_train.shape[1]\n",
    "X_mean = X_mean.reshape(X_train.shape[0],1)\n",
    "X_var_square = np.sum(np.square(X_train))/X_train.shape[1]\n",
    "\n",
    "X_train_norm = (X_train - X_mean)/ np.sqrt(X_var_square)\n",
    "X_test_norm = (X_test - X_mean)/ np.sqrt(X_var_square)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2529ca60-b6c9-4fe2-a99f-be646e21fa09",
   "metadata": {},
   "source": [
    "In order to predict if an image is zero (y=0) or another number (y=1),\n",
    "we take only part of the dataset and we make sure there are at least 30% of zeros."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "b46aad43-aa4c-42ef-9f44-88127ed58998",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "There are  0.100 % images of zeros\n",
      "The training set has shape (784, 500)\n"
     ]
    }
   ],
   "source": [
    "m_train = 500\n",
    "\n",
    "y_0 = (y_train[:,0:m_train]>0).astype(int)\n",
    "print('There are ', '{:.3f}'.format(1-y_0.sum()/m_train),'% images of zeros')\n",
    "\n",
    "X_0 = X_train_norm[:,0:m_train]\n",
    "print('The training set has shape', X_0.shape)\n",
    "\n",
    "y_0_test = (y_train[:,m_train: m_train +100]>0).astype(int)\n",
    "X_0_test = X_train_norm[:,m_train: m_train +100]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "c8271dfd-40b2-48aa-9d96-f26ae4cbe93e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The percentage of images of zeros is now  0.352\n",
      "The training set has shape (784, 694)\n"
     ]
    }
   ],
   "source": [
    "for i in range(X_train.shape[1]-2000, X_train.shape[1]):\n",
    "    if y_train[0,i] == 0:\n",
    "        y_0 = np.append(y_0,y_train[:,i].reshape(1,1), axis = 1)\n",
    "        X_0 = np.append(X_0,X_train_norm[:,i].reshape(784,1), axis = 1)\n",
    "\n",
    "print('The percentage of images of zeros is now ', '{:.3f}'.format(1-y_0.sum()/y_0.shape[1]))\n",
    "print('The training set has shape', X_0.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "884123cf-af8d-4ed7-9bc9-7005e72decca",
   "metadata": {},
   "source": [
    "### The model:\n",
    "Let's apply the model() function to the training and test sets X_0 and y_0 and plot the evolution of the cost function during the iterations of the algorithm. Note how this starts pretty high (0.7), but after few iterations it tends to zero."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "4dde9fc9-a225-4fb5-9a24-a54774694259",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAf+klEQVR4nO3de5gcZZ328e+dmUwOnCED5EiCBjG4EHCMsgLqK2riKYi8GjzLajau7CqIbpTFy7Miyou7gllWEZdV44FT1Ai6nlARyRACEiASI8iQQAYIBBJIMsPv/eOpmXT39Ex6klT3TOr+XFddXYdnun+p6fQ9T1XXU4oIzMysuEY0ugAzM2ssB4GZWcE5CMzMCs5BYGZWcA4CM7OCcxCYmRWcg8BsECS9T9JDkp6UdFAdX/djkr5er9ezYnEQ2LAk6S2S2rMP5HWSfirphF18znslnTzA9pHAhcArI2LviHhkV15vgNd5qaSO0nUR8bmIeE8er2fmILBhR9LZwEXA54BDgCnAJcDcnF/6EGA0sDLn1zGrKweBDSuS9gM+Bbw/Iq6KiE0RsS0ifhQRH87ajJJ0kaS12XSRpFHZtnGSfizpMUmPSvqtpBGSriAFyo+yXsZHKl73CGBVtviYpF9KmiopJDWXtPu1pPdk8++S9DtJX5K0QdJfJc0paXugpG9mNW6QdI2kvYCfAhOyOp6UNEHSJyT9T8nPvl7Syuzf8WtJzy3Zdq+kcyTdLulxSd+TNHr3/iZsT+IgsOHmeNJf5VcP0OZc4EXATOAYYBbwb9m2DwEdQCvpL/yPARERbwf+BrwuO+zzxdInjIg/A0dli/tHxP+psd4XkgJkHPBF4BuSlG27AhibPe/BwP+LiE3AHGBtVsfeEbG29AmzUPou8MHs37GUFGAtJc3eBMwGpgFHA++qsV4rIAeBDTcHAQ9HRNcAbd4KfCoi1kdEJ/BJ4O3Ztm3AeOCwrCfx28h3wK37IuK/IqIb+Fb22odIGk/6wF8QERuyWn5T43O+GfhJRPw8IrYBXwLGAH9f0ubfI2JtRDwK/IgUimZVOQhsuHkEGFd6OKaKCcB9Jcv3ZesALgBWAz+TtEbSwnzK7PVgz0xEbM5m9wYmA49GxIadeM6yf19EPAPcD0ys9rrA5uw1zapyENhw8wfgaeCUAdqsBQ4rWZ6SrSMinoiID0XE4cDrgLMlvTxrN9iewabscWzJukNr/Nn7gQMl7V9l247qKPv3ZYeaJgMP1PjaZmUcBDasRMTjwMeBiyWdImmspJGS5kjqOa7/XeDfJLVKGpe1/x8ASa+V9Ozsw3Mj0J1NAA8Bhw+ilk7Sh+/bJDVJOgN4Vo0/u450UvgSSQdk/4aTSuo4KDsxXs33gddIenn2ldYPAVuAG2ut3ayUg8CGnYi4EDibdAK4k/TX9ZnANVmTzwDtwO3An4Dl2TqA6cD/Ak+SeheXRMSvs22fJwXIY5LOqbGc9wIfJh2yOorBfRi/nXTO4m5gPenkLxFxNynM1mS1TCj9oYhYBbwN+A/gYVLP5nURsXUQr23WS74xjZlZsblHYGZWcA4CM7OCcxCYmRWcg8DMrOAGuihnSBo3blxMnTq10WWYmQ0rt9xyy8MR0Vpt27ALgqlTp9Le3t7oMszMhhVJ9/W3zYeGzMwKzkFgZlZwDgIzs4LLNQgkzZa0StLqaqM8SvqwpBXZdIekbkkH5lmTmZmVyy0IJDUBF5PGXJ8BnC5pRmmbiLggImZGxEzgo8BvsvHTzcysTvLsEcwCVkfEmmwwrMUMfE/Z00kDbZmZWR3lGQQTSaNC9uig/MYZvSSNJd1W78oc6zEzsyryDAJVWdffUKevA37f32EhSfMltUtq7+zs3Llq7rgDzjsPdvbnzcz2UHkGQQfprkk9JpHdJaqKeQxwWCgiLo2Itohoa22temHcjt19N3zmM/Dggztua2ZWIHkGwTJguqRpklpIH/ZLKhtld2F6CXBtjrXA2Oxugps3D9zOzKxgchtiIiK6JJ0JXA80AZdFxEpJC7Lti7KmbwB+FhGb+nmq3WPMmPT41FO5voyZ2XCT61hDEbEUWFqxblHF8uXA5XnWATgIzMz6UZwri31oyMysquIEgXsEZmZVFS8I3CMwMytTnCDoOTTkHoGZWZniBIEPDZmZVVWcIBg9Oj360JCZWZniBIGUegXuEZiZlSlOEICDwMysiuIFgQ8NmZmVKVYQjB3rHoGZWYViBYEPDZmZ9VG8IPChITOzMsUKAh8aMjPro1hB4ENDZmZ9FC8IfGjIzKxMsYLAh4bMzPooVhD40JCZWR/FCoKxY31oyMysQrGCwD0CM7M+ihcEXV2wbVujKzEzGzJyDQJJsyWtkrRa0sJ+2rxU0gpJKyX9Js96fHMaM7O+mvN6YklNwMXAK4AOYJmkJRFxZ0mb/YFLgNkR8TdJB+dVD1B+c5p99831pczMhos8ewSzgNURsSYitgKLgbkVbd4CXBURfwOIiPU51uO7lJmZVZFnEEwE7i9Z7sjWlToCOEDSryXdIukd1Z5I0nxJ7ZLaOzs7d76inkND/uaQmVmvPINAVdZFxXIz8HzgNcCrgPMkHdHnhyIujYi2iGhrbW3d+YrcIzAz6yO3cwSkHsDkkuVJwNoqbR6OiE3AJkk3AMcAf86lIgeBmVkfefYIlgHTJU2T1ALMA5ZUtLkWOFFSs6SxwAuBu3KryIeGzMz6yK1HEBFdks4ErgeagMsiYqWkBdn2RRFxl6TrgNuBZ4CvR8QdedXkHoGZWV95HhoiIpYCSyvWLapYvgC4IM86ejkIzMz6KNaVxT40ZGbWR7GCwD0CM7M+HARmZgVXzCDwoSEzs17FCoLmZhg50j0CM7MSxQoC8D0JzMwqFC8IfJcyM7MyxQsC9wjMzMo4CMzMCs5BYGZWcMULgpYW2Lq10VWYmQ0ZxQuCUaNgy5ZGV2FmNmQULwjcIzAzK1O8IHCPwMysTPGCwD0CM7MyxQuCUaMcBGZmJYoXBC0tPjRkZlaieEHgHoGZWZniBYF7BGZmZYoXBO4RmJmVyTUIJM2WtErSakkLq2x/qaTHJa3Ipo/nWQ/gHoGZWYXmvJ5YUhNwMfAKoANYJmlJRNxZ0fS3EfHavOroo6UFurvT1NRUt5c1Mxuq8uwRzAJWR8SaiNgKLAbm5vh6tRk1Kj368JCZGZBvEEwE7i9Z7sjWVTpe0m2SfirpqGpPJGm+pHZJ7Z2dnbtWVUtLenQQmJkB+QaBqqyLiuXlwGERcQzwH8A11Z4oIi6NiLaIaGttbd21qnp6BD5PYGYG5BsEHcDkkuVJwNrSBhGxMSKezOaXAiMljcuxJvcIzMwq5BkEy4DpkqZJagHmAUtKG0g6VJKy+VlZPY/kWJPPEZiZVcjtW0MR0SXpTOB6oAm4LCJWSlqQbV8EnAa8T1IX8BQwLyIqDx/tXj09Ah8aMjMDcgwC6D3cs7Ri3aKS+a8CX82zhj7cIzAzK1O8K4vdIzAzK1PcIHCPwMwMKGIQ+OujZmZlihcE7hGYmZUpXhC4R2BmVqZ4QeAegZlZmeIFgb8+amZWpnhB4K+PmpmVKV4QuEdgZlameEHgHoGZWZniBoF7BGZmQBGDwF8fNTMrU7wgaGoCyT0CM7NM8YJASr0C9wjMzIAiBgGk8wTuEZiZAUUNAvcIzMx6FTMI3CMwM+tVzCAYNcpBYGaWKWYQtLT40JCZWSbXIJA0W9IqSaslLRyg3QskdUs6Lc96erlHYGbWK7cgkNQEXAzMAWYAp0ua0U+784Hr86qlD/cIzMx65dkjmAWsjog1EbEVWAzMrdLun4ErgfU51lLOJ4vNzHrlGQQTgftLljuydb0kTQTeACwa6IkkzZfULqm9s7Nz1yvz10fNzHrlGQSqsi4qli8C/jUiugd6ooi4NCLaIqKttbV11ytzj8DMrFdzjs/dAUwuWZ4ErK1o0wYslgQwDni1pK6IuCbHutwjMDMrkWcQLAOmS5oGPADMA95S2iAipvXMS7oc+HHuIQDuEZiZlcgtCCKiS9KZpG8DNQGXRcRKSQuy7QOeF8iVvz5qZtYrzx4BEbEUWFqxrmoARMS78qyljL8+ambWq5hXFrtHYGbWq6YgkPR/a1k3bLhHYGbWq9YewUdrXDc8+GSxmVmvAc8RSJoDvBqYKOnfSzbtC3TlWViuer4+GpHuWGZmVmA7Olm8FmgHXg/cUrL+CeCsvIrKXUtLCoHubmjO9Xy5mdmQN+CnYETcBtwm6TsRsQ1A0gHA5IjYUI8CczFqVHrcssVBYGaFV+s5gp9L2lfSgcBtwDclXZhjXflqaUmPPk9gZlZzEOwXERuBU4FvRsTzgZPzKytnpT0CM7OCqzUImiWNB94E/DjHeurDPQIzs161BsGnSENF/CUilkk6HLgnv7Jy1tMjcBCYmdU2xERE/AD4QcnyGuCNeRWVu54egQ8NmZnVfGXxJElXS1ov6SFJV0qalHdxuXGPwMysV62Hhr4JLAEmkO4y9qNs3fDkHoGZWa9ag6A1Ir4ZEV3ZdDmwG24V1iBjxqTHzZsbW4eZ2RBQaxA8LOltkpqy6W3AI3kWlqu99kqPmzY1tg4zsyGg1iA4g/TV0QeBdcBpwLvzKip3DgIzs161jq/waeCdPcNKZFcYf4kUEMOPg8DMrFetPYKjS8cWiohHgWPzKakOHARmZr1qDYIR2WBzQG+PYPiO1uYgMDPrVeuH+ZeBGyX9EAjS+YLP5lZV3kaPTvchcBCYmdXWI4iI/yZdSfwQ0AmcGhFX7OjnJM2WtErSakkLq2yfK+l2SSsktUs6YbD/gJ0ipV6Bg8DMrPbDOxFxJ3Bnre0lNQEXA68AOoBlkpZkz9PjF8CSiAhJRwPfB46s9TV2iYPAzAyo/RzBzpgFrI6INRGxFVgMzC1tEBFPRkRki3uRDjvVh4PAzAzINwgmAveXLHdk68pIeoOku4Gf0M/XUSXNzw4dtXd2du6e6hwEZmZAvkFQ7a7wff7ij4irI+JI4BTS9Qp9fyji0ohoi4i21tbdNLKFg8DMDMg3CDqAySXLk4C1/TWOiBuAZ0kal2NN2+21l8caMjMj3yBYBkyXNE1SCzCPNIJpL0nPlqRs/jighXqNYeQegZkZkONFYRHRJelM0p3NmoDLImKlpAXZ9kWkr6S+Q9I24CngzSUnj/PlIDAzA3K+OjgilgJLK9YtKpk/Hzg/zxr65SAwMwPyPTQ0tDkIzMwABwHU6UiUmdlQVewg6O72fYvNrPCKHQTgw0NmVngOAgeBmRWcg8BBYGYF5yBwEJhZwTkIHARmVnAOAgeBmRWcg8BBYGYF5yBwEJhZwTkIHARmVnAOAgeBmRVccYNg7Nj06CAws4IrbhCMGAFjxjgIzKzwihsE4KGozcxwEDgIzKzwHAQOAjMrOAeBg8DMCi7XIJA0W9IqSaslLayy/a2Sbs+mGyUdk2c9feyzD2zcWNeXNDMbanILAklNwMXAHGAGcLqkGRXN/gq8JCKOBj4NXJpXPVWNHw/r1tX1Jc3Mhpo8ewSzgNURsSYitgKLgbmlDSLixojYkC3eBEzKsZ6+JkyAtWt932IzK7Q8g2AicH/Jcke2rj//APw0x3r6mjABtmyBDRt23NbMbA/VnONzq8q6qn96S3oZKQhO6Gf7fGA+wJQpU3ZXfSkIIPUKDjxw9z2vmdkwkmePoAOYXLI8CVhb2UjS0cDXgbkR8Ui1J4qISyOiLSLaWltbd1+FpUFgZlZQeQbBMmC6pGmSWoB5wJLSBpKmAFcBb4+IP+dYS3UOAjOz/A4NRUSXpDOB64Em4LKIWClpQbZ9EfBx4CDgEkkAXRHRlldNfYwfnx4dBGZWYHmeIyAilgJLK9YtKpl/D/CePGsY0JgxcMABDgIzK7RiX1kM279CamZWUA6CiRMdBGZWaA6CCRPggQcaXYWZWcM4CCZMSMNMPPNMoysxM2sIB8GECdDdDZ2dja7EzKwhHAS+lsDMCs5BMCkb5+7eextahplZozgIjjoKmppg+fJGV2Jm1hAOgrFjYcYMuOWWRldiZtYQDgKAtjZob/d9CcyskBwEAM9/fvrWUEdHoysxM6s7BwGkHgGkXoGZWcE4CACOPjqdMPZ5AjMrIAcBpFFIn/c8WLas0ZWYmdWdg6DHCSfA734Hmzc3uhIzs7pyEPQ49dQUAtdd1+hKzMzqykHQ46ST4KCD4Ic/bHQlZmZ15SDo0dwMp5wCP/4xbNnS6GrMzOrGQVDqtNPgiSdg6dIdtzUz20M4CEq9/OVw2GFw4YWNrsTMrG5yDQJJsyWtkrRa0sIq24+U9AdJWySdk2ctNRk5Ej70ofTtod//vtHVmJnVRW5BIKkJuBiYA8wATpc0o6LZo8C/AF/Kq45BO+OMdNL4859vdCVmZnWRZ49gFrA6ItZExFZgMTC3tEFErI+IZcC2HOsYnL32gnPOgZ/8xF8lNbNCyDMIJgL3lyx3ZOsGTdJ8Se2S2jvrcUvJs86C5zwH3v9+eOqp/F/PzKyB8gwCVVm3U+M8R8SlEdEWEW2tra27WFYNRo2Cr30N1qyBj3wk/9czM2ugPIOgA5hcsjwJGD43Bn7Zy1LP4KtfhcWLG12NmVlu8gyCZcB0SdMktQDzgCU5vt7ud/758OIXpxPI/haRme2hcguCiOgCzgSuB+4Cvh8RKyUtkLQAQNKhkjqAs4F/k9Qhad+8ahq0kSPhqqtg8mR4zWs8TLWZ7ZEUw+z2jG1tbdFe7xvI/O1vcOKJsGEDXHttOmxkZjaMSLolItqqbfOVxbWYMiUdGpo8GV71qnQieZgFqJlZfxwEtZo0KV1xfPLJ8E//BPPmpfscm5kNcw6CwTjggDQ66Wc/C9dcAzNmpG8UuXdgZsOYg2CwRoyAj30Mli+Hww+H00+H2bPh1lsbXZmZ2U5xEOyso45K5w0uugja2+G441IorFrV6MrMzAbFQbArmpvhAx9IVyCfey4sWQLPfS7MnQs33OBDRmY2LDgIdof99oPPfAb++lc47zy48UZ4yUtg5szUY1i/vtEVmpn1y0GwOx18MHzyk+m6g//8zzRm0VlnwcSJ6TaYV1/tQezMbMhxEORhzBiYPx9uvhnuuCOFwR//CKeeCq2t8MY3whVXwKOPNrpSMzNfWVw3XV3wy1+mr51eey2sXQtNTWkso5NPTrfJfMEL0rAWZma72UBXFjsIGuGZZ9K4Rddck25+c+ut6cTyPvukcwsnngjHHw9tbal3YWa2ixwEQ90jj8CvfgW/+EWa7rknrW9uhmOOSaHQEwzPfna6lsHMbBAcBMNNZyfcdFOa/vCHdK5h06a0bexYOProFBAzZ6bH5z0v9SbMzPrhIBjuurrSSedbb4XbboMVK9LjY49tbzNhQrq95nOeA0ceuf1xyhT3IMxswCBorncxthOam9Nf/zNnbl8Xkb6mumIF3HUX3H13mhYvLg+I0aNh6tQ0TZu2fb5nam0FVburqJkVhYNguJLgsMPSNHfu9vUR6QK2VatSMKxaBffem6abb+77ldWxY1MgTJqUehXVpkMP9beZzPZgDoI9jQSHHJKmk07qu33jRrjvvu3hcO+96YroBx6AO++Edeugu7vvc7a2pudsbR14OvhgOPBAH44yG0YcBEWz777wd3+XpmqeeSadrF67tu/00ENp2/Ll6bH0EFSpESPSsBsHHAD77z/wY8/8vvumE957750em/3WNKsX/2+zciNGbO9RHHvswG23bYOHH06hsH59euyZNmxIQdHzuHbt9uWnn95xHaNHbw+FysfKdXvvna636JnGji1frlw3erTPi5iVcBDYzhs5EsaPT9NgPP00PP54eVhs3AhPPAFPPln+WDr/+OPQ0VG+rqtr52ofPbp6YIwalaaWlu2PpfO7sm7kyNTTaW7ePl9tXc/ksLI6yTUIJM0GvgI0AV+PiC9UbFe2/dXAZuBdEbE8z5psCBg9Ok2HHLJrzxMBW7akayyeeipNmzdvn6+catm2dWuaf+yxNL9lS/+PeWtq6j8oag2VpqY0jRixfb6W5d3VZqCfkdL8iBHDY34PDubcgkBSE3Ax8AqgA1gmaUlE3FnSbA4wPZteCHwtezTbMWl7qNRbROqN7CgsSh+7utK0bdvA8zvaXsv8pk1pvrs7nffp7i6fdmad1R4cA021tOmv3XvfC2efvdv/WXn2CGYBqyNiDYCkxcBcoDQI5gL/Hemqtpsk7S9pfESsy7Eus10npb+6R46EvfZqdDX10RMKuyNYnnkmTRF71nx3d3rc0dTTfrDtDj00l19tnkEwEbi/ZLmDvn/tV2szESgLAknzgfkAU6ZM2e2FmlkNev7itT1Onr/VagfUKsezqKUNEXFpRLRFRFtra+tuKc7MzJI8g6ADmFyyPAlYuxNtzMwsR3kGwTJguqRpklqAecCSijZLgHcoeRHwuM8PmJnVV27nCCKiS9KZwPWkr49eFhErJS3Iti8ClpK+Orqa9PXRd+dVj5mZVZfrdQQRsZT0YV+6blHJfADvz7MGMzMbmL8CYGZWcA4CM7OCcxCYmRXcsLtVpaRO4L6d/PFxwMO7sZzdaajW5roGZ6jWBUO3Ntc1ODtb12ERUfVCrGEXBLtCUnt/9+xstKFam+sanKFaFwzd2lzX4ORRlw8NmZkVnIPAzKzgihYElza6gAEM1dpc1+AM1bpg6NbmugZnt9dVqHMEZmbWV9F6BGZmVsFBYGZWcIUJAkmzJa2StFrSwgbWMVnSryTdJWmlpA9k6z8h6QFJK7Lp1Q2o7V5Jf8pevz1bd6Ckn0u6J3s8oAF1Padkv6yQtFHSBxuxzyRdJmm9pDtK1vW7jyR9NHvPrZL0qjrXdYGkuyXdLulqSftn66dKeqpkvy3q94nzqavf31u99tcAtX2vpK57Ja3I1tdlnw3w+ZDveywi9viJNPrpX4DDgRbgNmBGg2oZDxyXze8D/BmYAXwCOKfB++leYFzFui8CC7P5hcD5Q+B3+SBwWCP2GXAScBxwx472UfZ7vQ0YBUzL3oNNdazrlUBzNn9+SV1TS9s1YH9V/b3Vc3/1V1vF9i8DH6/nPhvg8yHX91hRegS990+OiK1Az/2T6y4i1kXE8mz+CeAu0u05h6q5wLey+W8BpzSuFABeDvwlInb26vJdEhE3AI9WrO5vH80FFkfEloj4K2m49Vn1qisifhYRXdniTaQbP9VVP/urP3XbXzuqTZKANwHfzev1+6mpv8+HXN9jRQmC/u6N3FCSpgLHAn/MVp2ZdeMva8QhGNJtQn8m6ZbsPtEAh0R2s6Ds8eAG1FVqHuX/ORu9z6D/fTSU3ndnAD8tWZ4m6VZJv5F0YgPqqfZ7G0r760TgoYi4p2RdXfdZxedDru+xogRBTfdGridJewNXAh+MiI3A14BnATOBdaRuab29OCKOA+YA75d0UgNq6JfSne5eD/wgWzUU9tlAhsT7TtK5QBfw7WzVOmBKRBwLnA18R9K+dSypv9/bkNhfmdMp/4OjrvusyudDv02rrBv0PitKEAypeyNLGkn6JX87Iq4CiIiHIqI7Ip4B/oscu8T9iYi12eN64Oqshockjc/qHg+sr3ddJeYAyyPiIRga+yzT3z5q+PtO0juB1wJvjeygcnYY4ZFs/hbSceUj6lXTAL+3hu8vAEnNwKnA93rW1XOfVft8IOf3WFGCoJb7J9dFduzxG8BdEXFhyfrxJc3eANxR+bM517WXpH165kknGu8g7ad3Zs3eCVxbz7oqlP2V1uh9VqK/fbQEmCdplKRpwHTg5noVJWk28K/A6yNic8n6VklN2fzhWV1r6lhXf7+3hu6vEicDd0dER8+Keu2z/j4fyPs9lvdZ8KEyke6N/GdSkp/bwDpOIHXdbgdWZNOrgSuAP2XrlwDj61zX4aRvH9wGrOzZR8BBwC+Ae7LHAxu038YCjwD7layr+z4jBdE6YBvpr7F/GGgfAedm77lVwJw617WadPy45322KGv7xux3fBuwHHhdnevq9/dWr/3VX23Z+suBBRVt67LPBvh8yPU95iEmzMwKriiHhszMrB8OAjOzgnMQmJkVnIPAzKzgHARmZgXnILA9jqQbs8epkt6ym5/7Y9VeKy+STpH08Wz+ckmnVWmzWNL0POuwPZuDwPY4EfH32exUYFBB0HPR0ADKgqDktfLyEeCSHbT5WtbObKc4CGyPI+nJbPYLwInZ+PFnSWpSGqN/WTbg2T9m7V+ajQH/HdKFTki6Jht8b2XPAHySvgCMyZ7v26WvpeQCSXco3dPhzSXP/WtJP1S6N8C3s6tHkfQFSXdmtXypyr/jCGBLRDxcZdunsx7CCOC3wMnZ0Ahmg+Y3ju3JFpLGvX8tQPaB/nhEvEDSKOD3kn6WtZ0FPC/SUL4AZ0TEo5LGAMskXRkRCyWdGREzq7zWqaRB1I4BxmU/c0O27VjgKNIYML8HXizpTtLwCkdGRCi7aUyFF5OuYi0j6YvAfsC7I7siVNLq7LVvqXXnmPVwj8CK5JXAO5TuOvVH0mX7PcfWby4JAYB/kXQbaRz/ySXt+nMC8N1Ig6k9BPwGeEHJc3dEGmRtBemQ1UbgaeDrkk4FNvd9SsYDnRXrzgP2j4h/jPJhAdYDE3ZQo1lVDgIrEgH/HBEzs2laRPT0CDb1NpJeShp47PiIOAa4FRhdw3P3Z0vJfDfprmFdpF7IlaSbjFxX5eeeqvK6y4DnSzqwYv3orL3ZoDkIbE/2BOl2fz2uB96XDfOLpCOykVYr7QdsiIjNko4EXlSybVvPz1e4AXhzdh6ilXQbxH5HgczGm98vIpYCHyQdVqp0F/DsinXXkc59/KRntNjMEaRB0cwGzecIbE92O9CVHeK5HPgK6bDM8uyEbSfVb715HbBA0u2kER1vKtl2KXC7pOUR8daS9VcDx5NGpwzgIxHxYBYk1ewDXCtpNKk3cVaVNjcAX5ak0sNAEfGDLASWKN34fV/gqcjuYGU2WB591GwIk/QV4EcR8b8DtDkL2BgR36hfZbYn8aEhs6Htc6R7MQzkMbbf2Nxs0NwjMDMrOPcIzMwKzkFgZlZwDgIzs4JzEJiZFZyDwMys4P4/OQEsmsizJg8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "n_x = X_0.shape[0]    # input shape\n",
    "n_y = 1               # output: 0 if it is a zero, 1 otherwise\n",
    "\n",
    "layer_dimensions = [n_x, 3, n_y]\n",
    "num_iter = 20000\n",
    "\n",
    "param, costs = model(X_0, y_0, layer_dimensions, alpha = 0.075, num_iterations = num_iter)\n",
    "plot_cost(costs, num_iter)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5414a3bb-221a-420f-beb7-4b6a4a877bb3",
   "metadata": {},
   "source": [
    "#### It's time to test the model on new images!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "1987c150-2e35-4768-b727-2e965c3b9bb1",
   "metadata": {},
   "outputs": [],
   "source": [
    "def test_model(X_0, y_true, param, activ = ['relu','sigmoid']):\n",
    "\n",
    "    AL, _ = model_forward(X_0, param, activ)\n",
    "    \n",
    "    y_pred = (AL > 0.5).astype(int) # 1 if non zero\n",
    "\n",
    "    pred = (y_pred==y_true).astype(int)\n",
    "    \n",
    "    return pred.sum()/pred.shape[1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "4de80219-faa9-4230-acf7-ee6ee996918d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy of the train set: 1.0\n",
      "Accuracy of the test set: 0.99\n"
     ]
    }
   ],
   "source": [
    "print('Accuracy of the train set:', test_model(X_0, y_0, param))\n",
    "print('Accuracy of the test set:', test_model(X_0_test, y_0_test, param))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "150c28cb-ecb9-4a53-81e5-458169e70554",
   "metadata": {},
   "outputs": [],
   "source": [
    "def test_example(X, i):\n",
    "    \n",
    "    digit = X.loc[i].values\n",
    "    plt.imshow(digit.reshape(28,28), cmap = 'binary')\n",
    "    plt.axis('off')\n",
    "    plt.show()\n",
    "    \n",
    "    digit = X.loc[i].values.reshape((784,1))\n",
    "    digit_pred, _ = model_forward(digit, param, ['relu','sigmoid'])\n",
    "    \n",
    "    if np.squeeze(digit_pred) >0.5:  \n",
    "        prediction = 'non zero'\n",
    "    else: prediction = 'zero'\n",
    "    \n",
    "    print('The algorithm predicts a ' + prediction +  ' picture.')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "7cf03a46-1efa-4e25-9738-959e2b5965f5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAOcAAADnCAYAAADl9EEgAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8QVMy6AAAACXBIWXMAAAsTAAALEwEAmpwYAAAFnklEQVR4nO3dv29NfxzH8d5vDLTBwFIx119gMdi6NRISsaODJhLSoVNFWCViMGH2Y+hkt1gMSEcswm6pxVTTdxDnvD+c2/LS83iM3vnce5LbZz6JT845k+3t7Rkgz39/+wKAbuKEUOKEUOKEUOKEUPsac/+VC7tv0vWPdk4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4IJU4Ite9vX8AYbW1t9c5WV1fLtQ8fPtzpy/nB+vp672xtba1cOzs7u9OXM2p2TgglTgglTgglTgglTgglTgg12d7erublkG4bGxvl/Pbt272zzc3Ncu1kMhl0Tb+q+nu4d+9eufbq1as7fTlj0fmj2jkhlDghlDghlDghlDghlDghlDghlHPOXXDq1Kly/urVq95Z4/dofvbx48fLecuzZ896Z4cOHSrX3rp1q5xfvny5nI/4ljPnnPAvESeEEieEEieEEieEEieEEieE8mjMAa5cuVLO3759O/izP336VM6PHDlSzg8cODD4u2dmZmbu3LnTO1tZWSnXXr9+vZx/+fKlnN+8ebOcj42dE0KJE0KJE0KJE0KJE0KJE0KJE0I55+zw+fPncv7ixYty/u3bt3J+48aN3tm092NOq/r+/fv3l2tb96JWrz7kZ3ZOCCVOCCVOCCVOCCVOCCVOCCVOCOWcs8Pi4mI5//DhQzlvPVt2bW3tt68pQevdoK35sWPHdvJy9jw7J4QSJ4QSJ4QSJ4QSJ4QSJ4RylNLh/fv35bx1ZNC67Wvax1fuppcvX/bOWrfKtVy4cGGq9WNj54RQ4oRQ4oRQ4oRQ4oRQ4oRQ4oRQk8bjDOtnHe5RJ0+eLOdv3rwp57Ozs+X88ePHvbOlpaVy7W47fPhw7+zr169Tffb6+no5H/ErADsPzu2cEEqcEEqcEEqcEEqcEEqcEEqcEMo5Z4fWq+pOnz5dzjc3Nwd/97Vr18r5tOegz58/L+d3797tnbXuY22d77YeKTo/P1/O9zDnnPAvESeEEieEEieEEieEEieEEieEcs45wMbGRjm/ePFiOa/OURu/R/Os8ejRo+V8bm6unH/8+HHwd58/f76cP3nypJyPmHNO+JeIE0KJE0KJE0KJE0KJE0KJE0J5P+cA586dK+et88CnT5/2zt69e1euPXHiRDlfWVkp569fvy7nq6ur5Zw/x84JocQJocQJocQJocQJocQJoRyl7IKzZ88Onrcey3nw4MFB1/S/Bw8eTLWeP8fOCaHECaHECaHECaHECaHECaHECaGcc4aZ9hxzWq1Hc1Zaj93k99g5IZQ4IZQ4IZQ4IZQ4IZQ4IZQ4IZRzzpGZn58v59VjPVuP/FxeXh50TXSzc0IocUIocUIocUIocUIocUIocUIo55wjU71+kCx2TgglTgglTgglTgglTgglTgglTgglTgglTgglTgglTgglTgglTgglTgjllrGRab3ib5pXAG5tbQ1ey8/snBBKnBBKnBBKnBBKnBBKnBBKnBDKOefItF7jN80rAB89elTOFxcXyzk/snNCKHFCKHFCKHFCKHFCKHFCKHFCKOec7JiFhYW/fQl7ip0TQokTQokTQokTQokTQokTQokTQjnnHJn79++X8zNnzvTOlpaWyrWXLl0adE10s3NCKHFCKHFCKHFCKHFCKHFCqEnjlW/D3wcH/KrOZ47aOSGUOCGUOCGUOCGUOCGUOCGUOCGUOCGUOCGUOCGUOCGUOCGUOCGUOCGUOCFU69GYnfeZAbvPzgmhxAmhxAmhxAmhxAmhxAmhvgPRVb8+DtcgfQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The algorithm predicts a non zero picture.\n"
     ]
    }
   ],
   "source": [
    "test_example(X, 26839)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}