Labs/Lab_3B.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.8.5"
    },
    "varInspector": {
      "cols": {
        "lenName": 16,
        "lenType": 16,
        "lenVar": 40
      },
      "kernels_config": {
        "python": {
          "delete_cmd_postfix": "",
          "delete_cmd_prefix": "del ",
          "library": "var_list.py",
          "varRefreshCmd": "print(var_dic_list())"
        },
        "r": {
          "delete_cmd_postfix": ") ",
          "delete_cmd_prefix": "rm(",
          "library": "var_list.r",
          "varRefreshCmd": "cat(var_dic_list()) "
        }
      },
      "types_to_exclude": [
        "module",
        "function",
        "builtin_function_or_method",
        "instance",
        "_Feature"
      ],
      "window_display": false
    },
    "colab": {
      "name": "Lab_3B.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/tirtharajdash/BITS-G513_MetaLearning/blob/main/Labs/Lab_3B.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vzEAvqjX--_X"
      },
      "source": [
        "- This notebook has been adapted from NYU's Deep Learning Course taught by Yann LeCun & Alfredo Canziani. [ [Website](https://atcold.github.io/pytorch-Deep-Learning/) ]\n",
        "- See [https://www.cs.ryerson.ca/~aharley/vis/conv/](https://www.cs.ryerson.ca/~aharley/vis/conv/) for an interactive CNN demo.\n",
        "- More about convolutions in signals: [Blog](https://dspillustrations.com/pages/posts/misc/convolution-examples-and-the-convolution-integral.html)\n",
        "- Math behind convolutions: [Blog](https://medium.com/@pavisj/convolutions-and-backpropagations-46026a8f5d2c)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hHPPa73---_a"
      },
      "source": [
        "# Outline\n",
        "\n",
        "* Today we will show how to train a ConvNet using PyTorch\n",
        "* We will also illustrate how the ConvNet makes use of specific assumptions"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "c6XIO_GO--_b"
      },
      "source": [
        "# To perform well, we need to incorporate some prior knowledge about the problem\n",
        "\n",
        "* Assumptions helps us when they are true\n",
        "* They hurt us when they are not\n",
        "* We want to make just the right amount of assumptions, not more than that\n",
        "\n",
        "## In Deep Learning\n",
        "\n",
        "* Many layers: compositionality\n",
        "* Convolutions: locality + stationarity of images\n",
        "* Pooling: Invariance of object class to translations"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3bDu489V--_b"
      },
      "source": [
        "import torch\n",
        "import torch.nn as nn\n",
        "import torch.nn.functional as F\n",
        "import torch.optim as optim\n",
        "from torchvision import datasets, transforms\n",
        "import matplotlib.pyplot as plt\n",
        "import numpy\n",
        "\n",
        "# function to count number of parameters\n",
        "def get_n_params(model):\n",
        "    np=0\n",
        "    for p in list(model.parameters()):\n",
        "        np += p.nelement()\n",
        "    return np"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Z8uGl8d5--_b"
      },
      "source": [
        "def plot_data(X, y, d=0, auto=False, zoom=1):\n",
        "    X = X.cpu()\n",
        "    y = y.cpu()\n",
        "    plt.scatter(X.numpy()[:, 0], X.numpy()[:, 1], c=y, s=20, cmap=plt.cm.Spectral)\n",
        "    plt.axis('square')\n",
        "    plt.axis(np.array((-1.1, 1.1, -1.1, 1.1)) * zoom)\n",
        "    if auto is True: plt.axis('equal')\n",
        "    plt.axis('off')\n",
        "\n",
        "    _m, _c = 0, '.15'\n",
        "    plt.axvline(0, ymin=_m, color=_c, lw=1, zorder=0)\n",
        "    plt.axhline(0, xmin=_m, color=_c, lw=1, zorder=0)\n",
        "\n",
        "\n",
        "def plot_model(X, y, model):\n",
        "    model.cpu()\n",
        "    mesh = np.arange(-1.1, 1.1, 0.01)\n",
        "    xx, yy = np.meshgrid(mesh, mesh)\n",
        "    with torch.no_grad():\n",
        "        data = torch.from_numpy(np.vstack((xx.reshape(-1), yy.reshape(-1))).T).float()\n",
        "        Z = model(data).detach()\n",
        "    Z = np.argmax(Z, axis=1).reshape(xx.shape)\n",
        "    plt.contourf(xx, yy, Z, cmap=plt.cm.Spectral, alpha=0.3)\n",
        "    plot_data(X, y)\n",
        "\n",
        "plt.rc('figure', figsize=[10,10], dpi=100)\n",
        "device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-qGIDwrU--_b"
      },
      "source": [
        "# Load the Dataset (MNIST)\n",
        "\n",
        "\n",
        "We can use some PyTorch DataLoader utilities for this. This will download, shuffle, normalize data and arrange it in batches."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "oJD2eVWb--_b"
      },
      "source": [
        "input_size  = 28*28   # images are 28x28 pixels\n",
        "output_size = 10      # there are 10 classes\n",
        "\n",
        "train_loader = torch.utils.data.DataLoader(\n",
        "    datasets.MNIST('../data', train=True, download=True,\n",
        "                   transform=transforms.Compose([\n",
        "                       transforms.ToTensor(),\n",
        "                       transforms.Normalize((0.1307,), (0.3081,))\n",
        "                   ])),\n",
        "    batch_size=64, shuffle=True)\n",
        "\n",
        "test_loader = torch.utils.data.DataLoader(\n",
        "    datasets.MNIST('../data', train=False, transform=transforms.Compose([\n",
        "                       transforms.ToTensor(),\n",
        "                       transforms.Normalize((0.1307,), (0.3081,))\n",
        "                   ])),\n",
        "    batch_size=1000, shuffle=True)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ESDd_wgT--_c",
        "outputId": "4d418d9c-bb33-41cf-e64c-122bb6145982"
      },
      "source": [
        "# show some images\n",
        "plt.figure(figsize=(16, 6))\n",
        "for i in range(10):\n",
        "    plt.subplot(2, 5, i + 1)\n",
        "    image, _ = train_loader.dataset.__getitem__(i)\n",
        "    plt.imshow(image.squeeze().numpy())\n",
        "    plt.axis('off');"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAABOQAAAHYCAYAAADtQTdtAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAr5klEQVR4nO3deZRdZZkv/n2qTqUykABJSBhMmEIRCEOAMCQCkVG8V0EEjChXxJFZaRCu9GiLXmhtlVlFAYdutBEF2gEUBS5tBuY5E0PCFAIEEgKZqs4594/u9Vu/Xvq8RXaq3lOp+nz+/eZ99kPI2efUt/Zap9JoNAoAAAAAII+WZi8AAAAAAAOJQg4AAAAAMlLIAQAAAEBGCjkAAAAAyEghBwAAAAAZKeQAAAAAICOFHAAAAABkpJADAAAAgIyq7/QPHtFyQqM3F4H+6Pf1GyvN3qGnuAfA+usv9wCvf1h//eX1XxTuAVBGf7kHeP3D+nunr39PyAEAAABARgo5AAAAAMhIIQcAAAAAGSnkAAAAACAjhRwAAAAAZKSQAwAAAICMFHIAAAAAkJFCDgAAAAAyUsgBAAAAQEYKOQAAAADISCEHAAAAABkp5AAAAAAgI4UcAAAAAGSkkAMAAACAjBRyAAAAAJCRQg4AAAAAMlLIAQAAAEBGCjkAAAAAyEghBwAAAAAZKeQAAAAAICOFHAAAAABkpJADAAAAgIwUcgAAAACQkUIOAAAAADJSyAEAAABARgo5AAAAAMhIIQcAAAAAGSnkAAAAACCjarMXAKD3dB26TzJfcvraMHtk6g/DbM9ZJyfnbn3loDBrvfPB5FkAAID+zhNyAAAAAJCRQg4AAAAAMlLIAQAAAEBGCjkAAAAAyEghBwAAAAAZKeQAAAAAICOFHAAAAABkVG32AvynSjX9v6J1i9G9ct35520XZrWh9TDbdsdXknOHnl4Js5e/OSjMHpzys+Tc12pvh9n+N56bPDvhr2Ync9hY1afvFWaXXXtF8uyEtvjeE98BiuKhqdcl586fUguzL253QPIs0H+9ffz+YXbJP12dPPuVD388zBr3P156J2D9PP31qcl87kfjzx5tldYwO/j0zybnDrn53vRiABsZT8gBAAAAQEYKOQAAAADISCEHAAAAABkp5AAAAAAgI4UcAAAAAGSkkAMAAACAjKrNXqAvat1lpzBrtLclz740fbMwW33A22E2ctM4K4qiuGfPnyXz3H67angyv+SKo8Jszu7/GmbPdq5Ozr146RFhtvU9jeRZ2Jh1HjklzM6/6sdh1tE2KDm3XtTD7JnOzjBbUW9Pzt0rEa99375hNuTOx5Jz62vWJHP6j9XH7JfOR7WG2chrZ/X0OvSQV6bEvwv+yqIPZNwESHn5nGlhdteMf0qe7WykP3uEfJQHBhhPyAEAAABARgo5AAAAAMhIIQcAAAAAGSnkAAAAACAjhRwAAAAAZKSQAwAAAICMqs1eoBlq79k7mX/z+ivDrKOt5Nd4b4Q6G7Uw+7vLP5E8W307/t7yqTeeGWbDX+xKzm1/bXWYDb1/TvIsNFvriBFh9vbBE5Nnz/nWv4bZIUPeSpws/3uX69+YFmZ/uGpq8uyf/uGyMPv9978TZrv+JL4/FEVR7HDBrGRO//HSwel/u0N3XB6H1/bsLqynltYwaoyP38cPGzMvOfYPlfieBPSst8bVw2xky8D5eQh627r3Tknmiz8WvxZP2/vuMPvC5gtK77T7988Ks6FL4p/zi6Iolk9bG2bb/kv82W7Q7fd3v1g/5Ak5AAAAAMhIIQcAAAAAGSnkAAAAACAjhRwAAAAAZKSQAwAAAICMFHIAAAAAkJFCDgAAAAAyqjZ7gWZon/9SMn9gzbgw62hb2tPrbLBzlxwQZs+8NTp59vodfx5mK+qNMBt72czuF+sF8UbQ973wo23C7L59r8y4yTvzj2PuC7PbNpmWPHvKoiPD7Ifb3RFmI3Zd1v1iDAhffv+NyfySufG/MZqrdcdtw2ze9GvDbPK9JyXnbn3fY6V3Av7cWyfsH2Y3HXtp4mQlOfc7yyeG2R0fnhJmwxY/kZxbT6bQd7166tQwu/z89M8AU9prYdaSeL7q5EWHJ+futelzYfbIp1Ov/7TUTtNGnhhmI28vfcmNmifkAAAAACAjhRwAAAAAZKSQAwAAAICMFHIAAAAAkJFCDgAAAAAyUsgBAAAAQEbVZi/QDF1LXk7ml19yQph99ai3k2dbH90kzB45/fL0YgkXvbZHmD11+NAwqy1fkpz70amnh9mis+Nz2xePJOfCQNV16D5hdsPkK8KspRhU+pqnLD4szO6/Y5fk2cc+Fe905+rBYTbm/tXJuU+9MTHM2r52Z5i1VJJjGUDaKl3NXoGSqt9fVerc6qdH9PAmMLCtef9+yfzv/8+1YdbRVv4N+YfXHBVmWz45s/RcaKZKW/qz+prD9wyzm7709TDbutqenPupxUeE2eJv7Bxmw379cHLunUPHh9ndv+wIs5t2ujU5N+XNh0eF2cjSUzdunpADAAAAgIwUcgAAAACQkUIOAAAAADJSyAEAAABARgo5AAAAAMhIIQcAAAAAGSnkAAAAACCjarMX6ItGXjcrzLb491HJs7Vlr4fZpN0+GWZPHHxtcu6t35seZmOWz0yeTanMeiTMto//GmDAqk/fK5lfdu0VYTahLb7l1ot6cu7R844Ns9bj3w6zzf5nIzl31x+fGWYdVz4fZi3PP5Scu/k9cdb51VqY3bRH+l74yUPODrPWOx9MnqXvqR84OcwOGvwf+RahR203bFmpc+PuiO8NwPpbctKaZH7IkFTeGiYnLzo8OXfLS8v/bAJ91ZIzpyTze8+7NJG2h8kJT30gObfruM4wG/ranDBL/wRQFC99dp8wm7NT6r8l7berhofZhO/GP1t0lb7ixs0TcgAAAACQkUIOAAAAADJSyAEAAABARgo5AAAAAMhIIQcAAAAAGSnkAAAAACCjarMX2NjUXltW+mznm4NKn530sSfD7NWr468lL+q10teEgaqyz6Qwe+2vVifPdrTFr/MH1sbn/vjWrsm5y346LsxGvTErzDb9yezk3E0TWTO+fnxsa/y18EVRFMu+sCrMxtzZ09vQ2xa/f0iYjWkdmnET1kd1u/HJ/PiRt5aaO+TZN5K5TzTw56rv2ibMnjjouuTZzkb8qprbGZ977psdybnDijnJHPqqhZfvH2bzP3R58mw9ke3y+1PDbOJ5i5JzN6R/SDn1tFt6Ze5FXz05zDZ/Pv6ZZaDyhBwAAAAAZKSQAwAAAICMFHIAAAAAkJFCDgAAAAAyUsgBAAAAQEYKOQAAAADIqNrsBQaSXS5YEGan7H5Y8ux12/4hzKafcEaYDf/Z7O4XgwGmZejQZN71T2+G2eyJv0iefbZrXZj91YXnhtnm9zyXnDtm2CthVkue7F/222pxmC3KtwY9pDphZemza+Zt1nOLsF6e//awZP7u9nqY/eDNd8UHl8f3XhjIWiftHGZT/vXxXrnmjF+cHWY73uTnCzZOT//zAcl8/oeuDLMV9TXJsyfM+2iY7XxW3APUVpb/LNQyLH4/Xnb8Hsmzx2zy9XhuMSTMJt4Ydw9FURQTrp+VzPnvPCEHAAAAABkp5AAAAAAgI4UcAAAAAGSkkAMAAACAjBRyAAAAAJCRQg4AAAAAMlLIAQAAAEBG1WYvMJDUlq8Is2Wn7ZI8+9ytq8Psf1/0ozD70oePTc5tPLRpmI376qzEwUZyLvRlq6dPSua3T7yq9OxPf/6cMBt+8+ww6yp9RRiYxtxfb/YKfV7r6FHJfOlxHWE28sMvhNndHT/o5sqDw+TqKz8YZmOWzuxmLgxMi4+OX8s/H/VQ4mRrcu5Hn/5AmHVc/HSY1ZJToblax44Jsx8em/6MXy/izxYnzPto8uygIxYn5pbXMnnXMNvt2rlhdtHYy7qZ3B4m7374I2G28z/E1ywK94f15Qk5AAAAAMhIIQcAAAAAGSnkAAAAACAjhRwAAAAAZKSQAwAAAICMFHIAAAAAkFG12Qvwn+qPpL8++CNf/mKY/cvffyPMHj7gR+kLHxBHk4adGWY7XbMkObbrmUXp60IT7fGVh5N5S+J3FacsPix5dsjN95ZZaUBpq7SGWWcjfba10s0fYMBYPTJ+nQ7rpWvWD9ormTdaK2H2/OHtYbZu687k3JZBtTD73UGXh1lbvE5RFEXxci3e6W+fOTbMXq/Xk3OHtsT7jp2zMsy8uhmoXj9lajL/5alfT6RtYXLq89OTcztPju8BtVefS56FvqoyOP53PaU9fn/qzpCzB6Wvu+24MFt46rvC7MjDH0zOPWfM98JsfHVImKXfqYui1ojfdSs/Gx2fW76wm8msD0/IAQAAAEBGCjkAAAAAyEghBwAAAAAZKeQAAAAAICOFHAAAAABkpJADAAAAgIyqzV6Ad2bktbPC7Mz5Z4TZiItfSM69YYfbw+yJj18RZhPHfTo5d+cvx11vbeEzybPQE5b/r6lh9jdjv5E8Wy/irzV/4He7Js+OL2amF6PobMRfOV/v5kvab5sb//3vVKS/Np6+Z+2atjCrF43k2esu/FaY3Xrm5LIrJV0w6vvJvKWohNnqxrowe6kWvyaKoiiuePU9YXb4HV8Is80eiu9lRVEUW/1uaZhVFsefH16dOyQ5d2xrZ5g17nsseRb6q9ZJO4fZzIviz9z/aXCpa856YbtkPm7R46XmQl/WWLM2zOasjT93FEVR7N8ev3/dcsdPk2e7+wxb1h2rR4fZws74s9IhQ95Kzr1/XfwZYbMfxd0DPcsTcgAAAACQkUIOAAAAADJSyAEAAABARgo5AAAAAMhIIQcAAAAAGSnkAAAAACAjhRwAAAAAZFRt9gJsuMqfHg6zVcePSZ7dd8ZZYTbngkvDbN4h30/O/dh2R4bZigOTR6FHdA2Js01bBiXPzlrTHmY7/Oil9HWTaf/RMnRoMp/3jd0S6QNh8rFn3pecO/Hzz4ZZLXmSvmjCSQ+F2aT/c2by7Lh9X+zpdbp15ysdyfzV374rzEY90Rlmg267r5srx2c7ivu7ORtLvWZevGBamO3bPis596dvbVNyI+i/FlwYv292NnrnHWz8xem80StXheaqLX0lzP7+tE8nz37jO1eF2R7pHx+Kn7w5LswuuvvoMOu4fk1ybnXpijAbc8PrYXbIuD8m5558Z/x3sSGfLVg/npADAAAAgIwUcgAAAACQkUIOAAAAADJSyAEAAABARgo5AAAAAMhIIQcAAAAAGVWbvQC9K/W1z0VRFGMvi/M153eF2dBK+nufr9nuV2H2/mO/EM/95ZzkXMhhWW2TMOt6ZlG+RZqsZejQMJt/8e7Js/OOuSLMfrtq0zB76coJybnD35idzOk/tv/SrGavsN62Kp5r9go9ZujBr5Y++zd3HhdmHcW9pedCX1efvleYXTTl5l655hGPfyTMNrn/8V65JmysBt1+fzK/cPv9euW6G/Let/KYeKdfj78lzDob6WevhixK/zxPHp6QAwAAAICMFHIAAAAAkJFCDgAAAAAyUsgBAAAAQEYKOQAAAADISCEHAAAAABlVm70AG65+4OQwe/qEwcmzu01eFGZDK+W/Cvny1+OvfR96S/rrpqHZzvvTCWHWUTyQcZPeV58ev1Zf+avVYTZ3yhXJuYc9NiPMhh31TJgNL2Yn5wJ937a3NJq9AjTFV6//Xpjt1lb+dXHekoPDbNMT3wizWukrAn1F15D4GarORvwqrxf15Nztr38uvmb3a9FDPCEHAAAAABkp5AAAAAAgI4UcAAAAAGSkkAMAAACAjBRyAAAAAJCRQg4AAAAAMlLIAQAAAEBG1WYvwH+qTNktmS84e1CYXfPuH4bZwYPXld4pZW2jM5nPfn37OKwv6eFt4C+oxFFLN7+LuPTAG8LsyqKj7EZNsfgfpybzmz7+zTDraIvvO3vfe3Jy7tbHPpleDAD6mb0GxZ8vOhu10nNnXbd3mI15Y2bpuUDfN/yns+Pwn/PtQe/whBwAAAAAZKSQAwAAAICMFHIAAAAAkJFCDgAAAAAyUsgBAAAAQEYKOQAAAADIqNrsBfqb6vbbhtnTp2wdZv8w46fJucdt8lrpncq6cOmUMLv70gOSZzf/4ayeXgfWTyOO6kU9eXT6kGVh9oXr90me3fG6eHbbyyvDbOn0LZJzR854IczOGv+HMHvf0AeSc299e2yYffyxo8Js9HeHJecC/VdrJf373Dc62sJsy9/29DaQz/M/3y2Zt1Ue7pXrbnVX/HNArVeuCPQVKz+S+rk7/Tmfvs8TcgAAAACQkUIOAAAAADJSyAEAAABARgo5AAAAAMhIIQcAAAAAGSnkAAAAACCjarMX6Iuq240PsxX7bJU8O+MfbwuzUzf7Remdyjp3Seprkoti1lVTwmzk9feG2eb1WaV3gr5ucCW+Nc494jvJs/9x0OAwW7h2yzA7ZdNF3e5VxudfOiiZ3zZzcpjt9PnZPbwN0B/UGvX0H/DrXjZi9el7hdm3J/8kebazUQuzFfU1Ybbvb7+QnDtx8ZPJHOi/VuzgTbU/838XAAAAADJSyAEAAABARgo5AAAAAMhIIQcAAAAAGSnkAAAAACAjhRwAAAAAZKSQAwAAAICMqs1eoLdUt9oyzF6/dljy7Gnb3x1mJw5fWnqnDXHmiweG2YNXTw6z0T9/PDl35MpZZVeCPm3sXa+E2QWfm5o8e8mW5V8XBw9eF2YHDl5Ueu5Da+Pfn5x492fDrOOUB5Jzdypml94J4C9Zte+qZq8Apa0ZOSjMDhz8djenW8Pk9lXjw6zjs/clp9a7uSrQf21zd/ye2nZmfM/pbPTGNvQ0T8gBAAAAQEYKOQAAAADISCEHAAAAABkp5AAAAAAgI4UcAAAAAGSkkAMAAACAjKrNXiBl3XunpPNzXg+zCyf8JsyOHNLdV5b3jqW11WF28K3nJs9O/Jt5YTZy+aww8zXpDFS1BU+H2cITtkue3fWss8LsyQ9fXnalpIm/OT2Z73xV/JXnHQ890NPrAIRaK36fCwA5VP70cJhd/+aYMDtx+IvJuasmbRVmg55/odu96Bk+UQEAAABARgo5AAAAAMhIIQcAAAAAGSnkAAAAACAjhRwAAAAAZKSQAwAAAICMFHIAAAAAkFG12QukLPpgui9csPuNvXLdK5fvGGaX3n1k8mylVgmziRc9G2Y7LZ2TnFtLpsD66HpmUTKfcE6cH33Ovj27zH/pKO5L5o1euSrAX7b2ji3CrDa5nnETyGvEwy+H2VkvHJo8+51xd/f0OgChb333+DA78bxLk2e3+tunwmzZ8j3ig7Mf7XYv3jlPyAEAAABARgo5AAAAAMhIIQcAAAAAGSnkAAAAACAjhRwAAAAAZKSQAwAAAICMKo1G4x39wSNaTnhnfxD4//y+fmOl2Tv0FPcAWH/95R7g9Q/rr7+8/ovCPQDK6C/3AK//vqt19KgwG3RTNXn2ZxN+FWbTHzkxzEZ+9NXk3NryFcl8oHinr39PyAEAAABARgo5AAAAAMhIIQcAAAAAGSnkAAAAACAjhRwAAAAAZKSQAwAAAICM0t+FCwAAAECfUnttWZitO25U8uwu//y5MJt7+HfD7OiJn0ovNfvRdM5/4wk5AAAAAMhIIQcAAAAAGSnkAAAAACAjhRwAAAAAZKSQAwAAAICMFHIAAAAAkJFCDgAAAAAyqjZ7AQAAAAB6Ru21Zcl8p5Pj/Ohi38TJR0tuxF/iCTkAAAAAyEghBwAAAAAZKeQAAAAAICOFHAAAAABkpJADAAAAgIwUcgAAAACQUaXRaDR7BwAAAAAYMDwhBwAAAAAZKeQAAAAAICOFHAAAAABkpJADAAAAgIwUcgAAAACQkUIOAAAAADJSyAEAAABARgo5AAAAAMhIIQcAAAAAGSnkAAAAACAjhRwAAAAAZKSQAwAAAICMFHIAAAAAkJFCDgAAAAAyUsgBAAAAQEYKOQAAAADISCEHAAAAABkp5AAAAAAgI4UcAAAAAGSkkAMAAACAjBRyAAAAAJCRQg4AAAAAMlLIAQAAAEBGCjkAAAAAyEghBwAAAAAZKeQAAAAAICOFHAAAAABkpJADAAAAgIwUcgAAAACQkUIOAAAAADJSyAEAAABARgo5AAAAAMhIIQcAAAAAGSnkAAAAACAjhRwAAAAAZKSQAwAAAICMFHIAAAAAkJFCDgAAAAAyUsgBAAAAQEYKOQAAAADIqPpO/+ARLSc0enMR6I9+X7+x0uwdeop7AKy//nIP8PqH9ddfXv9F4R4AZfSXe4DXP6y/d/r694QcAAAAAGSkkAMAAACAjBRyAAAAAJCRQg4AAAAAMlLIAQAAAEBGCjkAAAAAyEghBwAAAAAZKeQAAAAAICOFHAAAAABkpJADAAAAgIwUcgAAAACQkUIOAAAAADJSyAEAAABARgo5AAAAAMhIIQcAAAAAGSnkAAAAACAjhRwAAAAAZKSQAwAAAICMFHIAAAAAkJFCDgAAAAAyUsgBAAAAQEYKOQAAAADISCEHAAAAABkp5AAAAAAgI4UcAAAAAGSkkAMAAACAjBRyAAAAAJCRQg4AAAAAMqo2ewEAimLBdfsk82ff+4Mw++brO4TZHR+ekpxbe3JBejEAAIAmGvWnzcOspdJInn112vIe3qbneEIOAAAAADJSyAEAAABARgo5AAAAAMhIIQcAAAAAGSnkAAAAACAjhRwAAAAAZKSQAwAAAICMqs1egN7VOmpkMq9sOiLMnjtu6zBbM7qRnDvhy4+EWX3VquRZ6K9aJ+0cZrcccmXybGejLczO2Hx+mP18jyOTc4c/mYyBHlLZZ1KY1QelP469+J5hYfbEWVeFWWej1v1imR32+PFhNuyYJcmz9TVrenod6BMq7e3JfNX79gyzPf46/sy9cN+1pXcCyGnBD6Yk8/vGXxpmU+85I3l2h+LhMitl4Qk5AAAAAMhIIQcAAAAAGSnkAAAAACAjhRwAAAAAZKSQAwAAAICMFHIAAAAAkFG12QvwzrTsNjHMFn5pSJh9cveZybnnjrq99E4pu4w9Ncx2+sQDvXJN6PNefDmMzl7wkeTR30+6qae3AdZTY+qeyXzhJwaF2bcOvSHM2ipdybmHD1kZZp2N+Her9aKenNsMv9/t38Js8o8/mTy7/WkvhVnttWWld4Jma91idDK/88rvhNk9a+If576+/QeSc7ueXZxeDKAHLbh6vzC778hvJc+urDfCbMTdcR/S13lCDgAAAAAyUsgBAAAAQEYKOQAAAADISCEHAAAAABkp5AAAAAAgI4UcAAAAAGQUf082Pa6y7+5h9tQ5rcmzdx14RZht0doeZi3ddK6/XrV5mD2zdkyYnbH5/OTcHx98TZh9Zd+Tw6xx32PJubAxqy1fEWaLX9gpfXhSDy8DrLfGRa8n83kTf5Fpk/7p4WnXJvP37n96mLX/ellPrwMbhYMGd4XZV8ePTJ5teXZxT68DEHrPXnPDbHjLoOTZ0xcfFWajvzur9E7N5gk5AAAAAMhIIQcAAAAAGSnkAAAAACAjhRwAAAAAZKSQAwAAAICMFHIAAAAAkJFCDgAAAAAyqjZ7gY1N6xZbJPMFl24TZv8+7aow26GtrZsrt3eT/2XXvTkumd983IFhVm+PdzrjV/OTc6e018Js9dghYTY4ORU2bq1jx4TZQbssyLgJUMaLd6XfU4uJ5ebOWpN+j//kbz4Th5XEwUa5fYqiKA7YO74nXbfd78oPBnpUa8XzFbAxW33Mfsl89LnPhtnaGa1h1rXk5dI7bYhXTp8WZpeM/VaY/eTNbZNz3/jS+DBrKZZ1v1gf5Q4OAAAAABkp5AAAAAAgI4UcAAAAAGSkkAMAAACAjBRyAAAAAJCRQg4AAAAAMqo2e4GNzYsn7ZTMn5h+aSJt69ll/stP3hwXZjd/MP7a4aIoitr8BWFW2WtS6Z2Av2D4sDD6HyPv65VLvrJPJZlv9mhHmNWejO8PMBCNv/j+ZH7sv51Yam5lXWcy3+nZOaXmbojlo0eF2R2zhyfPHj5kZalrHvrYjGQ+4s4nwqxe6oqw8as14n/9nUPTP+q19/QywHo76eJfJfNTRjwfZofvc1qYDf7Vy6V32hAnn/GbMJvcHt91PvOVY5NzR94zq/ROfZkn5AAAAAAgI4UcAAAAAGSkkAMAAACAjBRyAAAAAJCRQg4AAAAAMlLIAQAAAEBGCjkAAAAAyKja7AU2NtscvahX5v78rS2T+TcXHBZmY89vhFlt/sLSO72x+4jSZ4E/V3vq2TD7m3+fkTx73IlXlrrmEx+9LJnvteLzYTbuyQWlrgn9VaNzXTKvzX8q0ya9b+mHOsJs90G3dHO6vdQ1X3ppZDLfZNUzpebCQPXKPm3JfNxvMy0ChJas2yyZ14vFYdY1pNLD23SvPn2vZH7MJpeHWWdjSJh1Dc7/39IXeEIOAAAAADJSyAEAAABARgo5AAAAAMhIIQcAAAAAGSnkAAAAACAjhRwAAAAAZFRt9gIbnc+0J+NdzzgrzMb9vhZmw554OTl39OIFYRZP3TCrxg7Mrx6GZtjxvNnpP3Binj2AgePV06aG2cST5oXZ2Nb0Z6Gydjn/2WTeW593oNkanZ3JfEHnmjDraBscZqu3X1d6J6DnLLxs/zD75ajLk2evXt4RZpvNfjHMurpfK9S62aZh9tp5byfPbl2NPyOc89K0MBv7gweScxvJdOPlCTkAAAAAyEghBwAAAAAZKeQAAAAAICOFHAAAAABkpJADAAAAgIwUcgAAAACQUbXZC2xsak89m8wnnJPOIxvytcS9pXPflc1eAfgvbZXWMOvsr98DDnTrlTOnhdnJp/0mefakEd8Is+Etg0rvlPKVV/cOs8badb1yTejraktfSeZnPz0jzG6beEtPrwOU0LrzhDD78fuvDrNVjc7k3F/89ZFhNuT5e7tfrISFV20fZo/vfU3y7B2rh8dz911beqf+yhNyAAAAAJCRQg4AAAAAMlLIAQAAAEBGCjkAAAAAyEghBwAAAAAZKeQAAAAAICOFHAAAAABkVG32Arwzz/3dtDDrGtqID1a6GZw4+qGdZnVzOHbmC+8JsyG3PVhmHRjQOhu1MKsX9YybwMDVOmnnZL7glM3DbPqBj/f0OkVRFMWvxl0eZt3fGwaVuuZTnV3JfMbV54bZ+F8uDbP6yqdL7QMAva3x7snJ/CM/+FWYTWmPP8dPvO3zybkdN9+bzMtadNHUMLv/4G8mTqYrpAu+/8kw26aY2d1aA44n5AAAAAAgI4UcAAAAAGSkkAMAAACAjBRyAAAAAJCRQg4AAAAAMlLIAQAAAEBG6e+sZb21jhgRZmv22ynM2r60NDn30YmXl9qnrdKazDsb8Vcwp9y5emgyf+Gz48Os0TW31DUBoLc13j05zD5x3S+TZ48Z9loPb/NO5P/d6tlPzUjm21wyM8zKfeoAythk5KpmrwB9SqVtUDJfcuaUMLv/vPTP46mfuzsb8Xv1hyY/mJx76yVTw2zClx8Js5YtxyTnHv0/ZodZa1EJs8kzP5mcO/7i+DMAf84TcgAAAACQkUIOAAAAADJSyAEAAABARgo5AAAAAMhIIQcAAAAAGSnkAAAAACCjarMX6Isq7e1htm767smz51z14zA7ZMgfwmxpbW1y7p2rNw+zv1twTJjdMOn65Nytq/F/a8rgls5k/syHNwuzHeYPDrP6mjWl9gGA3tZaNJJ5SxN+z9lWaQ2zzvS6pd22yy+T+UEfOyPMNv2X2T29DhC4ae9rkvlZxbszbQJ9w8unTknm9553aZjVu5mdes/90ZvbhNnXtpyTnPu1k+L8wsP3D7MjNv1tcu4hQ94Kszlr45/Xx5/wWHIu68cTcgAAAACQkUIOAAAAADJSyAEAAABARgo5AAAAAMhIIQcAAAAAGSnkAAAAACAjhRwAAAAAZFRt9gLN0DJ4cDJfNmOvMLvna5eVvu6kG84Ks3fdWUuebf/1fWE2aqu3wuyG2/dJzj131OPJPLJ/e2cyf/QT8d/T1OfPDrOxP3okObe+alV6Mein2iqtYdbZKD93xLRXyh+Gfqjyp4fD7AcfPCp59n9/YlSYjb99XZi1ru7qdq/esPBTbWE276irM24CpDz/H+PicGK+PWBj8OqpU8Ns5gXfTp5dWY9/xn2yc1jy7F+f97kwG7ws/gzwh68tSs69brvfhdnXtpwTZi3dPHtVT2RTBsX7nvPU3OTcS4/7UHzNR9JnByJPyAEAAABARgo5AAAAAMhIIQcAAAAAGSnkAAAAACAjhRwAAAAAZKSQAwAAAICMqs1eoLdU2tvDbN4390ienXfMZaWve8z8D4ZZx9efCbPa0leSc6vj3hVme976XJh9cdSTybkr6vFXGu9/07lhttXE9L5/2P1nYTbrb+O/3xknvj8597XLdg+zwcvir6nuTutdD5Y+Czl0NmphVk9+cXna3XveEGZHH/Cp+ODsR0tfEzZWtScXJPMdzs+0SA/ZZeEWcXhUvj2AtE2eb5Q6N7ySPte6a0eYdXe/g75q14/PDbNb3x6bPPu1750YZlv988zk2aHFnPRigWXnpruJcy4/KMy+tfU9pa7ZndZKJcy++NhxybNbP5LuH/jvPCEHAAAAABkp5AAAAAAgI4UcAAAAAGSkkAMAAACAjBRyAAAAAJCRQg4AAAAAMqo2e4ENUanG68//9p5hNu/oK5NzX+haG2ZHf/f85Nntrn06zLqWvhJmnYfvk5y72yUPhdnfj3kgzK57c9vk3B//9QfCbMIvZodZ6+hRybnvOeKsMHt7xoow++Ve1yTnvuuy9mSe8qu3452/17FD6bmQw8Q/fjrMnjz0e71yzQWfHRRmHfHtAdhILP3QhGavALwDLV3lzrVWKsm8PqSt3GDowx64fdcwe/2no5Nnt5o/s6fX6dbqsYOT+Vlb/DGRxq/hA/7xzOTc0Y+8ncwj4556MZnXSk0duDwhBwAAAAAZKeQAAAAAICOFHAAAAABkpJADAAAAgIwUcgAAAACQkUIOAAAAADJSyAEAAABARtVmL7Ahnv/ifmE27+hLw+ylrrXJuSdc/MUw2+7mZ5JnXz90+zBrnDQ8zH6+W7xvURTFFq3tYTbpp2eFWcf3XkvOHTp/TjKP1F5blsxH3BDnI26Izx1/+vnJuWOPX5zMk87dLBE+UX4uZNC+YEgcHppvD9gYVNrj98zlJ+yVPLv5LfH7QX3lytI7NcOSc6cl81vO/qdEGv8dAnltfv2sMPvO+duG2ambpj83LzxnUJhNOKn7vaAvGv/lmWFWy7jH/1/rFluE2QvHdSXPTmiL34//ZeVWYTb6u/F9Y0M06++wv/KEHAAAAABkpJADAAAAgIwUcgAAAACQkUIOAAAAADJSyAEAAABARgo5AAAAAMio2uwFNsTVn7mq1LnBlXT+gVP/b5htc/YbybMnj/j3MisVRRF/nXFRFMWkfz07zCZ86b4wq3Wlv0a5rxlzVfw11UVRFI1y/8v/y4sbchiaatxX4tfGDR/bJsw+NnxJ6Ws+e9T3w+x9e56YPFt/ZG7p68I7seYD+4XZpuc9F2Z3T7g8OffY+xL/tuev7Hav3lDdasswe/H4HcLsZ2d9Izl362r6s0dkaW1tMm9b3Sg1F1h/35j93jA76rBvJ892fG5BmNXLLgT8mYXnTgizuYddljw7a21bmP3b0QclTj7d3Vr0AZ6QAwAAAICMFHIAAAAAkJFCDgAAAAAyUsgBAAAAQEYKOQAAAADISCEHAAAAABlVm73Ahvi/b00Ms/3bHwuzka3tybkXjn647ErF++d9KMyem/WuMNvh5yuScyc88UCYNbq6ul8M6Leuf25amJ046cbSczsbpY9Cr3vvV+8Os3NHPV567rwLR8ThW/uXnrshPjJtVpjdPObXYVYv2kpf8+RF7w2zp67bOXl21C/ifYF8akUlmddXr8m0CfR/rbt2hNlXjv1pmNUa6Q/cp9x6aphNWDC7+8Xo0zwhBwAAAAAZKeQAAAAAICOFHAAAAABkpJADAAAAgIwUcgAAAACQkUIOAAAAADJSyAEAAABARtVmL7AhZh6ydZjt/7FDw2zFnuuSc6uvtoVZx3deTJ99+ZUw227N82FWT04FiK29fss4/Hq+PaA/mHv4d5u9wnqKf7c6a0178uRn5nw8zCZ8ZmGYjXp7VvdrAU23Y3VIMl92yn5hNuoHXuewPj78i7vC7NhN4o5g79mnJOdO+MLssiuxEfCEHAAAAABkpJADAAAAgIwUcgAAAACQkUIOAAAAADJSyAEAAABARgo5AAAAAMio2uwFNkRt2ethNvaymXG2Adfs2oCzAL1h84fje+GVb+ycPHvG5vN7eh3I4o9nvzvMfnT6fmH2yLuv7Y11NshP3hyXzJd0bhZm1z4Y/z1MuKaWnLvDnx4Os3ryJNBXXDc9vqe9UV+dPDv60bfCrFF6IxiYvnrLcWF24kmXhdmQ34zojXXYSHhCDgAAAAAyUsgBAAAAQEYKOQAAAADISCEHAAAAABkp5AAAAAAgI4UcAAAAAGSkkAMAAACAjKrNXgCADVN7ckGY3b7biOTZ24t9S151bslz0DNa73owzLa/d2iY7XP255Nzf/i5b4fZboMqybOHPjYjzFbctWWYbfuzF5Nzu55dHGY7FQ8kzwL92xfnHh9mx2/7UPJsy9trw6xWeiMYmHa4YFaYHX1B/Hl7VBGfo//zhBwAAAAAZKSQAwAAAICMFHIAAAAAkJFCDgAAAAAyUsgBAAAAQEYKOQAAAADIqNrsBQAAelJ91aow2+bimcmzF168X+nrblI8UyrrKn1FYKAb+f4FYfbHYlg3p+OzAPQ+T8gBAAAAQEYKOQAAAADISCEHAAAAABkp5AAAAAAgI4UcAAAAAGSkkAMAAACAjBRyAAAAAJCRQg4AAAAAMlLIAQAAAEBGCjkAAAAAyEghBwAAAAAZKeQAAAAAICOFHAAAAABkpJADAAAAgIwUcgAAAACQkUIOAAAAADJSyAEAAABARgo5AAAAAMhIIQcAAAAAGSnkAAAAACAjhRwAAAAAZFRpNBrN3gEAAAAABgxPyAEAAABARgo5AAAAAMhIIQcAAAAAGSnkAAAAACAjhRwAAAAAZKSQAwAAAICMFHIAAAAAkJFCDgAAAAAyUsgBAAAAQEb/D+NTogDKh63bAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 1600x600 with 10 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kQFQNAqT--_c"
      },
      "source": [
        "# Create the model classes"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "CjZ_ZFgd--_c"
      },
      "source": [
        "class FC2Layer(nn.Module):\n",
        "    def __init__(self, input_size, n_hidden, output_size):\n",
        "        super(FC2Layer, self).__init__()\n",
        "        self.input_size = input_size\n",
        "        self.network = nn.Sequential(\n",
        "            nn.Linear(input_size, n_hidden), \n",
        "            nn.ReLU(), \n",
        "            nn.Linear(n_hidden, n_hidden), \n",
        "            nn.ReLU(), \n",
        "            nn.Linear(n_hidden, output_size), \n",
        "            nn.LogSoftmax(dim=1)\n",
        "        )\n",
        "\n",
        "    def forward(self, x):\n",
        "        x = x.view(-1, self.input_size)\n",
        "        return self.network(x)\n",
        "    \n",
        "class CNN(nn.Module):\n",
        "    def __init__(self, input_size, n_feature, output_size):\n",
        "        super(CNN, self).__init__()\n",
        "        self.n_feature = n_feature\n",
        "        self.conv1 = nn.Conv2d(in_channels=1, out_channels=n_feature, kernel_size=5)\n",
        "        self.conv2 = nn.Conv2d(n_feature, n_feature, kernel_size=5)\n",
        "        self.fc1 = nn.Linear(n_feature*4*4, 50)\n",
        "        self.fc2 = nn.Linear(50, 10)\n",
        "        \n",
        "    def forward(self, x, verbose=False):\n",
        "        x = self.conv1(x)\n",
        "        x = F.relu(x)\n",
        "        x = F.max_pool2d(x, kernel_size=2)\n",
        "        x = self.conv2(x)\n",
        "        x = F.relu(x)\n",
        "        x = F.max_pool2d(x, kernel_size=2)\n",
        "        x = x.view(-1, self.n_feature*4*4)\n",
        "        x = self.fc1(x)\n",
        "        x = F.relu(x)\n",
        "        x = self.fc2(x)\n",
        "        x = F.log_softmax(x, dim=1)\n",
        "        return x"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Oyhzj6vV--_d"
      },
      "source": [
        "## Running on a GPU: device string\n",
        "\n",
        "Switching between CPU and GPU in PyTorch is controlled via a device string, which will seemlessly determine whether GPU is available, falling back to CPU if not:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Jtgews3z--_d"
      },
      "source": [
        "accuracy_list = []\n",
        "\n",
        "def train(epoch, model, perm=torch.arange(0, 784).long()):\n",
        "    model.train()\n",
        "    for batch_idx, (data, target) in enumerate(train_loader):\n",
        "        # send to device\n",
        "        data, target = data.to(device), target.to(device)\n",
        "        \n",
        "        # permute pixels\n",
        "        data = data.view(-1, 28*28)\n",
        "        data = data[:, perm]\n",
        "        data = data.view(-1, 1, 28, 28)\n",
        "\n",
        "        optimizer.zero_grad()\n",
        "        output = model(data)\n",
        "        loss = F.nll_loss(output, target)\n",
        "        loss.backward()\n",
        "        optimizer.step()\n",
        "        if batch_idx % 100 == 0:\n",
        "            print('Train Epoch: {} [{}/{} ({:.0f}%)]\\tLoss: {:.6f}'.format(\n",
        "                epoch, batch_idx * len(data), len(train_loader.dataset),\n",
        "                100. * batch_idx / len(train_loader), loss.item()))\n",
        "            \n",
        "def test(model, perm=torch.arange(0, 784).long()):\n",
        "    model.eval()\n",
        "    test_loss = 0\n",
        "    correct = 0\n",
        "    for data, target in test_loader:\n",
        "        # send to device\n",
        "        data, target = data.to(device), target.to(device)\n",
        "        \n",
        "        # permute pixels\n",
        "        data = data.view(-1, 28*28)\n",
        "        data = data[:, perm]\n",
        "        data = data.view(-1, 1, 28, 28)\n",
        "        output = model(data)\n",
        "        test_loss += F.nll_loss(output, target, reduction='sum').item() # sum up batch loss                                                               \n",
        "        pred = output.data.max(1, keepdim=True)[1] # get the index of the max log-probability                                                                 \n",
        "        correct += pred.eq(target.data.view_as(pred)).cpu().sum().item()\n",
        "\n",
        "    test_loss /= len(test_loader.dataset)\n",
        "    accuracy = 100. * correct / len(test_loader.dataset)\n",
        "    accuracy_list.append(accuracy)\n",
        "    print('\\nTest set: Average loss: {:.4f}, Accuracy: {}/{} ({:.0f}%)\\n'.format(\n",
        "        test_loss, correct, len(test_loader.dataset),\n",
        "        accuracy))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "os0dYjX_--_d"
      },
      "source": [
        "# Train a small fully-connected network"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Q-HJo0RT--_d",
        "outputId": "ca1caae1-157f-49a1-a7b3-5415734c8946"
      },
      "source": [
        "n_hidden = 8 # number of hidden units\n",
        "\n",
        "model_fnn = FC2Layer(input_size, n_hidden, output_size)\n",
        "model_fnn.to(device)\n",
        "optimizer = optim.SGD(model_fnn.parameters(), lr=0.01, momentum=0.5)\n",
        "print('Number of parameters: {}'.format(get_n_params(model_fnn)))\n",
        "\n",
        "for epoch in range(0, 1):\n",
        "    train(epoch, model_fnn)\n",
        "    test(model_fnn)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Number of parameters: 6442\n",
            "Train Epoch: 0 [0/60000 (0%)]\tLoss: 2.327711\n",
            "Train Epoch: 0 [6400/60000 (11%)]\tLoss: 1.796309\n",
            "Train Epoch: 0 [12800/60000 (21%)]\tLoss: 1.068493\n",
            "Train Epoch: 0 [19200/60000 (32%)]\tLoss: 0.821787\n",
            "Train Epoch: 0 [25600/60000 (43%)]\tLoss: 0.829244\n",
            "Train Epoch: 0 [32000/60000 (53%)]\tLoss: 0.729538\n",
            "Train Epoch: 0 [38400/60000 (64%)]\tLoss: 0.524437\n",
            "Train Epoch: 0 [44800/60000 (75%)]\tLoss: 0.489661\n",
            "Train Epoch: 0 [51200/60000 (85%)]\tLoss: 0.484068\n",
            "Train Epoch: 0 [57600/60000 (96%)]\tLoss: 0.412636\n",
            "\n",
            "Test set: Average loss: 0.4192, Accuracy: 8801/10000 (88%)\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vmgrjUBb--_e"
      },
      "source": [
        "# Train a ConvNet with the same number of parameters"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5pO2kHh7--_e",
        "outputId": "9984f2b3-3785-4ca0-89f3-fc29c42ef457"
      },
      "source": [
        "# Training settings \n",
        "n_features = 6 # number of feature maps\n",
        "\n",
        "model_cnn = CNN(input_size, n_features, output_size)\n",
        "model_cnn.to(device)\n",
        "optimizer = optim.SGD(model_cnn.parameters(), lr=0.01, momentum=0.5)\n",
        "print('Number of parameters: {}'.format(get_n_params(model_cnn)))\n",
        "\n",
        "for epoch in range(0, 1):\n",
        "    train(epoch, model_cnn)\n",
        "    test(model_cnn)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Number of parameters: 6422\n",
            "Train Epoch: 0 [0/60000 (0%)]\tLoss: 2.301894\n",
            "Train Epoch: 0 [6400/60000 (11%)]\tLoss: 2.140326\n",
            "Train Epoch: 0 [12800/60000 (21%)]\tLoss: 0.619331\n",
            "Train Epoch: 0 [19200/60000 (32%)]\tLoss: 0.543590\n",
            "Train Epoch: 0 [25600/60000 (43%)]\tLoss: 0.346163\n",
            "Train Epoch: 0 [32000/60000 (53%)]\tLoss: 0.249778\n",
            "Train Epoch: 0 [38400/60000 (64%)]\tLoss: 0.259461\n",
            "Train Epoch: 0 [44800/60000 (75%)]\tLoss: 0.089456\n",
            "Train Epoch: 0 [51200/60000 (85%)]\tLoss: 0.070798\n",
            "Train Epoch: 0 [57600/60000 (96%)]\tLoss: 0.209238\n",
            "\n",
            "Test set: Average loss: 0.1979, Accuracy: 9392/10000 (94%)\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_LxMfRnH--_e"
      },
      "source": [
        "# The ConvNet performs better with the same number of parameters, thanks to its use of prior knowledge about images\n",
        "\n",
        "* Use of convolution: Locality and stationarity in images\n",
        "* Pooling: builds in some translation invariance\n",
        "\n",
        "# What happens if the assumptions are no longer true?\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "09tbgPP1--_e",
        "outputId": "14ec2244-9432-4d23-db9b-1e88d3fc29ff"
      },
      "source": [
        "perm = torch.randperm(784)\n",
        "plt.figure(figsize=(16, 12))\n",
        "for i in range(10):\n",
        "    image, _ = train_loader.dataset.__getitem__(i)\n",
        "    # permute pixels\n",
        "    image_perm = image.view(-1, 28*28).clone()\n",
        "    image_perm = image_perm[:, perm]\n",
        "    image_perm = image_perm.view(-1, 1, 28, 28)\n",
        "    plt.subplot(4, 5, i + 1)\n",
        "    plt.imshow(image.squeeze().numpy())\n",
        "    plt.axis('off')\n",
        "    plt.subplot(4, 5, i + 11)\n",
        "    plt.imshow(image_perm.squeeze().numpy())\n",
        "    plt.axis('off')"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAABNsAAAOeCAYAAAA6JxaAAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAA9hAAAPYQGoP6dpAABw0klEQVR4nO3dd5xdVbk//j0lFQKEhEAyaYRkSKghBZIAckUQxBiMgIp4VcSCCChi1+u9Kl71oqIgRZQiFlREipEioPIFkkAKoYZUEpIhtNBJITPn/P7w1/T67Aw765wp5/3+9zN7rWfmnLXOPs/s12vVlcvlDAAAAADYdvUdXQAAAAAAdBeabQAAAACQiGYbAAAAACSi2QYAAAAAiWi2AQAAAEAimm0AAAAAkIhmGwAAAAAkotkGAAAAAIlotgEAAABAIo3t/cEj608oV7IQ6GpuK11T19E1VJM9AP5RLe0B1j/8o1pa/1lmD4B/Vkt7gPUP/6i969+TbQAAAACQiGYbAAAAACSi2QYAAAAAiWi2AQAAAEAimm0AAAAAkIhmGwAAAAAkotkGAAAAAIlotgEAAABAIpptAAAAAJCIZhsAAAAAJKLZBgAAAACJaLYBAAAAQCKabQAAAACQiGYbAAAAACSi2QYAAAAAiWi2AQAAAEAimm0AAAAAkIhmGwAAAAAkotkGAAAAAIlotgEAAABAIpptAAAAAJCIZhsAAAAAJKLZBgAAAACJaLYBAAAAQCKabQAAAACQiGYbAAAAACSi2QYAAAAAiTR2dAEApNN6+MQwW3fa5jB7YOrPw2z/OR8MsyEX9gyzhr8uDDMAAIDuypNtAAAAAJCIZhsAAAAAJKLZBgAAAACJaLYBAAAAQCKabQAAAACQiGYbAAAAACTS2NEF1Kq6xvhP37DLwIrMueSzI8OsrW8pzEbs8UyY9T2tLsye+kHPMFs46bdh9lzba2F20DVnh9noz8wNM+guSocdkJuff/mPw2x0j3jfiXeALLt/6hVhtmRSW5h9buSUnFGB7uy14w8Ks+/+z8Vh9s13fyDMyvMf3qaagDduxblTw2zx++J7jh51DWH2ptM+FmZ9rr+vfYUBdHKebAMAAACARDTbAAAAACARzTYAAAAASESzDQAAAAAS0WwDAAAAgEQ02wAAAAAgkcaOLqCzaBg3JszKvXqE2ZOH7RRmG6e8FmY77xhnd+3/2zDrCDdv6Bdm3/3x0WF2776/DrPHt2wMs+88fWSYDbmrHGbQXWx566Qw+/xFv8i9trlHzzArZaUwW7llS5i9VOoVZgfEUbb5bZPDrM9fHwqz0qZN8aDUhI3HHhhnAxrCbOfL51SiHAp4ZlL8/9xvrnpHFSsBtuaps6aF2d/e8z9htqUc33PkcjsP1ABPtgEAAABAIpptAAAAAJCIZhsAAAAAJKLZBgAAAACJaLYBAAAAQCKabQAAAACQSGNHF1BNbf82Icx+cOWFYdbco+Cx1l3IlnJbmH3tgg+FWeNr8dndU685Pcz6tbSGWa/nNoZZ3/n3hhl0Ng077BBmr71pbJiddd6vw+zNfV7dyqzF/ody5QvTwuyOi6aG2T3/dX6Y3fazS8Jsr1/G+8OoL8wJM2rDk2+K38d993gxvvDy9LWQo74hjMrD48/ytwx6LMzuqIv3IqAyXh1WCrOd67v/9yCoptePmhRmq0+K1+InJtyZO+6n+y8tVM++PzsjzPqui7/rvzhtc5iN+FV8H9fz1vntK6wb8GQbAAAAACSi2QYAAAAAiWi2AQAAAEAimm0AAAAAkIhmGwAAAAAkotkGAAAAAIk0dnQB1dRryZNhtmDTsDBr7vF0Jcop5Ox1U3Lzla8ODLMr9/h9mL1Uio/13fX82VsvLKG4Euha1l7VFGbzJl9YxUq27huD5oXZLdtPC7OTV701zH4+8vYw22Gv9e0rjJr09enXhNl3F8fvOaqrYY8RYfbYYZeH2fj73h9mQ+Y9tE01Af/aqyccFGbXzvxRzpV1YXLJi2PD7PZ3Twqz7VY/EmalnEqgq3j21KlhdsHn4+8Ak3q1hVn9Vp6T+uCqI8LsgB2fCLMHPpK3/mN59Uzb+cQw2/nWQtN1SZ5sAwAAAIBENNsAAAAAIBHNNgAAAABIRLMNAAAAABLRbAMAAACARDTbAAAAACCRxo4uoJpa1z0VZhd894Qw+9bRr4VZw4Pbh9kDp13QvsL+yTnP7Rdmy4/om3tt24vrwux9U08Ls1VnxmPunj2QOyfUstbDJ4bZ1eN/HGb1Wc9C8528+i25+fzbx4XZQ6fE9fx1Y+8wGzR/Y5gtf2FsmPX477+GWX1dGEHWo661o0ugHRp/tqHQdRtX7JC4EiDLsmzT9APD7D+/fXmYNfco9qH8858eHWa7PTq70JjQmdT1iO/XNx2xf5hd+6Vzw2xIY68wO2X1kWG2+nt7hlmWZdl2f1oUZn/tOzzM7ryuOcyuHXNj7pyRlxcNCLOdC43YNXmyDQAAAAAS0WwDAAAAgEQ02wAAAAAgEc02AAAAAEhEsw0AAAAAEtFsAwAAAIBEGju6gM5i5yvmhNkuf4yPrm1b/3yY7b3Ph8PskTfFx2/feOlhYTboxeLHaNfNeSDMdo9/fah5pcMOCLPzL/9xmI3uEW+xpawUZjMemxlmDce/FmZZlmU7vb0cZnv94vQwa75wTZjVr7k/zPrfFdey5VttYXbtfvEe+OE3nxlmDX9dGE9Il1I6ZHyYHdr77uoVQmEjt1tf6Lpht8d7A1DcuvdvCrM394mzLGsIkw+uOiLMdvtR8e8l0BWsO31SmN332R/lXNkrTE5Y/o4waz1uS5j1fe7enPmyLP4GkGVPfmximN07Ju/3iN28oV+Yjf5J/L2itdBsXZMn2wAAAAAgEc02AAAAAEhEsw0AAAAAEtFsAwAAAIBENNsAAAAAIBHNNgAAAABIpLGjC+gK2p4rdrT9lpd7Frpu75MeDbNnL46P5s6yLMtKbYXmhFpXN3HvMHvuMxvDrLlHvM4XbI7n+8ure4XZ+t8MC7MBL8yJB82ybMdfzo2znOuqfQz3rg3xkejrP70hzAb9tRLV0BFWT+8TZoMa+laxEvI0jhweZsfvfGOhMfs8/kKYuYuBWOPQptz8kUOvCLMt5Xh1Ld4Sj/nED5rDbLvs3tx6oCtYdsFBYbbkXReEWSlnzHG3nRpmYz+7KsyK9h225tRP3JB8zHO+9cEw678m//tKrfBkGwAAAAAkotkGAAAAAIlotgEAAABAIpptAAAAAJCIZhsAAAAAJKLZBgAAAACJNHZ0Ad3ZuC8sDbOT931LmF0x4o4wO+yET+bO2e+3c7deGNSo+r59w6z1f14Os7lj/xBmj7e+Hmaf+fLZYdb/rifCbNB2z4RZW5h0HwcOXh1mq6pXBhXWOPqVQtdtemyntIWQa80Ptwuzg3uVwuyyl4fGg74Y77dQ6xr23jPMJv364YrM+Z4/nBlme1zruwVd34rvTwmzJe+6MMxeKm0KsxMee1+Y7XlG3Adoe6XY/U/9dvHncZZl2frj9wuzY7c/Nx436xNmY6+Jew+jr5yTWw+ebAMAAACAZDTbAAAAACARzTYAAAAASESzDQAAAAAS0WwDAAAAgEQ02wAAAAAgkcaOLqA7a3vxpTBb/4lxYfbEjRvD7IvnXJU755fePTPMyvfvGGbDvpVzdG+5nDsndBUbD9s7zG4de1GhMT/yqbPCrN/1c8OstdBswKD5pY4uodNqGDggzJ4+rjnMdn732jC7s/mynBl7h8nFF74zzAY9PTtnTKhtq2fE6/j3A+7fytUNYfK+Fe8Is+bvrAiztq3MCJ1Fw66DwuznM+P7/FIW31ec8Nj7wqznkatzxiymfvxeYbbP5Ytzrz1n1/Nz0l5hcvCi94bZnv8Vz2lv2DpPtgEAAABAIpptAAAAAJCIZhsAAAAAJKLZBgAAAACJaLYBAAAAQCKabQAAAACQSGNHF1CrSg/Ex+i+9+ufC7Nf/ef3csddNOWqOJwSR3tvd3qYjfnpujBrXbkqtx7oTPb75qIwq8/538PJq98SZn2uv29bSurWetQ1hNmWcnxdQ11OSM3buHO8VrerwHylQw8Is3JDXZitOaJXmL0+ZEuY1fdsC7M/H3pBmGVZlvWIy8meaovr+Y+VM8Ps+VIpzPrWx7Xueu8rYWaFU+ueP3lqmF136rk5V/bIHffUNYeF2ZYPxntA27NP5I4LXUFd7/g9PqlX/HmVp8+ZPeP5RgwLs2WnDg2ztx6xMMzOGnRpmA1v7BNmWZZl8ad1lrWV40/eut8OjK97cVnunOTzZBsAAAAAJKLZBgAAAACJaLYBAAAAQCKabQAAAACQiGYbAAAAACSi2QYAAAAAiWi2AQAAAEAijR1dAP/bzpfPCbPTl3wy99odvrM2zK4edWuYPfKBH4fZ2GEfCbM9vx73a9uWrQwzqJQX/31qmH111++FWSnrGWYL/rxXmA3PZrevsBq0pdwWZqWsFGa3LI7/3mOyhdtUE53H5k09wqyUlcPsii+fF2Y3nj5+W0r6l74w4GdhVp/VhdnG8uth9mRbvDZ+/Oy/hdkRt386zLIsy3a6P97HBv/56TCrWx3fOzy7uE+Y7dqwJczK8x4KM6gFDXvvGWazz4nvu7Osd+E556wdGWbDVj1ceFzoCsqbNofZvZvje46DesWfZTfc/pswy7uXLer2jQPDbNmW+N4oy7LszX1eDbP5r8f3BztdFfce2DaebAMAAACARDTbAAAAACARzTYAAAAASESzDQAAAAAS0WwDAAAAgEQ02wAAAAAgkcaOLoA3pu6eRbn5huMHhdnk95wRZvd+4Udh9tibfxZmJ418a5i9dEgYQcW09omzHevjY6/nbOoVZqOuejKer11VdW31ffuG2WPf2yfnygVhctLKt4XZ2E89HmZtObPRtYx+//1htve3Tw+zYZNbKlFO6K/PNIfZszcPDbMBj2wJs563zMuZMb6uOZufc12+vLXT8oVpYTa515ww+82rTYXrge5u6Zfjz84t5cp8mg3/TpyVKzIjdB5tTz8TZv/5iY+E2fcuuSjM9ou/OmS/fHlYmJ1z54wwa75yU5g1Pv1SmA26+vm4mCzL3jzsL2H2wb/Gv/+23FuQz5NtAAAAAJCIZhsAAAAAJKLZBgAAAACJaLYBAAAAQCKabQAAAACQiGYbAAAAACTS2NEFkFbekce7nh9nmz7fGmZ96+Izj386claYTZ/56XjM6+4NM+gI69u2D7PWlauqV0gHqe/bN8yWfGffMHvs2B+H2c0bdgyzJy8cHWb9XpgbZtSG3b80p6NLaJfB2RMdXUISfd/0bKHrvvrX48KsObuvaDnQZZQOOyDMzpl0ffL5jnz4vbn59vMfTj4ndAc9b50fZl/e/cDk8xX9DHzl2LiWPw2/IffaLeX4Oao+q+Lv81SOJ9sAAAAAIBHNNgAAAABIRLMNAAAAABLRbAMAAACARDTbAAAAACARzTYAAAAASKSxowvgjSkdMj43X3FC7zDbZ/yqMOtbV+w44Auej48873tDfMQydDafveeEMGvOFlSxksopHRav12c+szHMFk/6cZi95aH3hNl2R68Ms37Z3DADuoYRN5Q7ugToUN+68tIw26dHsfXx2XVvCrMdT3wh99q2QjMCnUVrn/hZqC3l/BVeykphtvuVT8Rzbr0sCvJkGwAAAAAkotkGAAAAAIlotgEAAABAIpptAAAAAJCIZhsAAAAAJKLZBgAAAACJNHZ0AbWqbtI+Ybb0zJ5h9tODf5477pt6v164psjm8pYwm/v87vGFpXXJa4Gtqouj+pz/L/zokKvD7MKseVsqqqrV35gaZtd+4Adh1twj3ncm3PfBMBsy89H2FQYA3cwBPeP7ii3ltkJjzrliQpgNemF2oTGBrqHfb+bG4ferVwdpeLINAAAAABLRbAMAAACARDTbAAAAACARzTYAAAAASESzDQAAAAAS0WwDAAAAgEQaO7qArq5x9xFhtuLkIWH2X+/5TZgdt/1z21RTEV9+elKY3fmjKWHW/+dzKlEOFFeOo1JWCrPD+qwPs09fOTHM9rgiHrPHU6+E2dOH7RJmO79nbZidMfyOMMuyLHtb3wVhduNru4bZBx46OswG/mS73DmB7quhLv6/7AvNPcJst5srUQ1U35rf7xNmPeoWJZ9v8N/i7wFtyWcDOpNX3ht/786y+B6fzsmTbQAAAACQiGYbAAAAACSi2QYAAAAAiWi2AQAAAEAimm0AAAAAkIhmGwAAAAAk0tjRBXQWjSOHh9lLEweH2Xu+cUuYnbrTH7appiLOXhcfFzznoklhtvOV94VZ/9KcbaoJuoLedfF2uPjIS8Ls7kN7h9myzbuF2ck7rmpXXW/Up548NMxumT0+zMZ8am4FqgG6urZyKQ79y5ZuonTYAWH2w/G/DLMt5bYwe6m0Kcwm3/zpMBu7+tEwA7q3l0b5YO1OvJoAAAAAkIhmGwAAAAAkotkGAAAAAIlotgEAAABAIpptAAAAAJCIZhsAAAAAJNLY0QWk1jh4tzB7/vLtwuwTu98ZZif2e3qbanqjTm85JMwWXjw+99qBv384zHZ+ZU7RkqDL2PVvz4TZFz4+Ncy+u1ux9fGm3q+H2SG9VxUa8/7N8f9BTrzzY7nXNp+8IMzGZHML1QPwr2yYvKGjS4AkNu3cM8wO6f1azpUNYXLrhuFh1vyxeWFWypkN6N6a7ow/V3ucHu83WZZlW8qpq2FbebINAAAAABLRbAMAAACARDTbAAAAACARzTYAAAAASESzDQAAAAAS0WwDAAAAgEQaO7qAyOtHTYqzs54Psy+PvinM3ton7+ju9J5u2xhmb7rx7DAb+9XHwmznF+fkzum4cGpd29IVYbbshJFhttcZZ4TZo+++YFtK+pfG3nRamO15UXzsd/P9C5LXAhBpqPN/WQCohrp7FoXZlS8Pyr32xH4tYbZh78Fh1nPN2q3WRTHuoAAAAAAgEc02AAAAAEhEsw0AAAAAEtFsAwAAAIBENNsAAAAAIBHNNgAAAABIpLGjC4isemfcB1y67zXJ57vwxT3C7Ed3vjXM6trqwmzsOY+H2Zin7w2ztjABtkXrylVhNvqsOJtx1uTktTRn88KsnHw2gNjm23cJs7bxpSpWAh1jh0VPhdkZaw8Ps0uG3VmJcgD+l/N+cnxufuJnfxRmg/9jeZitf3G/eNC5D261LmKebAMAAACARDTbAAAAACARzTYAAAAASESzDQAAAAAS0WwDAAAAgEQ02wAAAAAgkbpyudyuHzyy/oT2/SDUiNtK19R1dA3VZA+Af1RLe4D1D/+oltZ/ltkD4J/V0h5g/XcODQMH5OY9r20Ms9+OnhVmhz1wYpjt/L5nw6ztxZdy6+nO2rv+PdkGAAAAAIlotgEAAABAIpptAAAAAJCIZhsAAAAAJKLZBgAAAACJaLYBAAAAQCLx+bAAAAAAdKi259bn5q8fNyDMxn3/42G2+IifhNmMsafEE859MLcePNkGAAAAAMlotgEAAABAIpptAAAAAJCIZhsAAAAAJKLZBgAAAACJaLYBAAAAQCKNHV0AAAAAAMW0Pbc+zMZ8MM5mZJNzRn1wGyrCk20AAAAAkIhmGwAAAAAkotkGAAAAAIlotgEAAABAIpptAAAAAJCIZhsAAAAAJFJXLpc7ugYAAAAA6BY82QYAAAAAiWi2AQAAAEAimm0AAAAAkIhmGwAAAAAkotkGAAAAAIlotgEAAABAIpptAAAAAJCIZhsAAAAAJKLZBgAAAACJaLYBAAAAQCKabQAAAACQiGYbAAAAACSi2QYAAAAAiWi2AQAAAEAimm0AAAAAkIhmGwAAAAAkotkGAAAAAIlotgEAAABAIpptAAAAAJCIZhsAAAAAJKLZBgAAAACJaLYBAAAAQCKabQAAAACQiGYbAAAAACSi2QYAAAAAiWi2AQAAAEAimm0AAAAAkIhmGwAAAAAkotkGAAAAAIlotgEAAABAIpptAAAAAJCIZhsAAAAAJKLZBgAAAACJaLYBAAAAQCKabQAAAACQiGYbAAAAACSi2QYAAAAAiWi2AQAAAEAije39wSPrTyhXshDoam4rXVPX0TVUkz0A/lEt7QHWP/yjWlr/WWYPgH9WS3uA9Q//qL3r35NtAAAAAJCIZhsAAAAAJKLZBgAAAACJaLYBAAAAQCKabQAAAACQiGYbAAAAACSi2QYAAAAAiWi2AQAAAEAimm0AAAAAkIhmGwAAAAAkotkGAAAAAIlotgEAAABAIpptAAAAAJCIZhsAAAAAJKLZBgAAAACJaLYBAAAAQCKabQAAAACQiGYbAAAAACSi2QYAAAAAiWi2AQAAAEAimm0AAAAAkIhmGwAAAAAkotkGAAAAAIlotgEAAABAIpptAAAAAJCIZhsAAAAAJKLZBgAAAACJNHZ0AQC1aOkVE8Ps8aMuC7MfPD8qd9zb3z0pzNoeXbr1wgAAAKpswD39w6y+rhxmz057sQLVbDtPtgEAAABAIpptAAAAAJCIZhsAAAAAJKLZBgAAAACJaLYBAAAAQCKabQAAAACQSGNHF0BaDQN2DrO6HXcIsyeOGxJmmwbGx+yO/voDYVbasCHMoBY07L1nmN3w5gvDbEu5R5h9sv+S3Dl/v99bw6zfo7mXAgnVTdw7zEo949uvln/bLsweOeOi3Dm3lNu2XliVvOXh48Nsu2PXhVlp06ZKlAOdSl2vXmG24W37h9l+X4nvu7Msy5ZN3ly4JoBqWHrZpDCbN/xHYTb1rk+G2ahs0baUVDGebAMAAACARDTbAAAAACARzTYAAAAASESzDQAAAAAS0WwDAAAAgEQ02wAAAAAgkfjseTpM/T5jw2zZl/rkXvvhfWeH2dkDbi1cU2TcrqeG2ZgPLUg+H3QpLU+F0ZlL3xtmt+19bSWqAQooT90/zJZ9qGeYnXf41WHWo641zI7o80qYbSnn/4+0lJVy82q6bZ/fhdn4X3w4zHb/xJNh1vbc+m2qCTqLhl0GhtlfL7wkzO7alP/V7dzd3xFmrY+v3nphAAksvfjAMJv31vPC7JVSOcx2uDO/D9IZebINAAAAABLRbAMAAACARDTbAAAAACARzTYAAAAASESzDQAAAAAS0WwDAAAAgETyz49mm9RN3jfMlp/VEGZ/O+THYbZLQ6/cOetz+qd/2tA/zFZuHhRmn+y/JMx+8aafhtk3J38wzMrzHgoz6C7aXnwpzFavHRNfuHcFigEKKZ/zfJg9NvYPVayk+1g07fIwO+qg08Ks15/WV6Ic6DIO7d2am39r+M5hVv/46tTlAPxL/3bA4jDrV98zzE5bfXSYDfzJnG2qqSN4sg0AAAAAEtFsAwAAAIBENNsAAAAAIBHNNgAAAABIRLMNAAAAABLRbAMAAACARBo7uoCuoGGXXcJs6Y+awuyP0y4Ks1E9euTM2Ks9Zf1LV7w8LMyuP+6QMCv1iuv55KwlYTapV1uYbdy1T5j1DhPoPhp2HRRmh45bWsVKgKJa/hZ/rmZji405Z1P8Of/hmz4aX1i3lYHLxeqZMiHej64Y+edigwLJNdR5TgK6uo3HHhhmA89+PMw2v6chd9zWdU8VrqmIZ06bFmbf3fW8MPvlyyPC7IUvDQ+z+mx9+wrrROzYAAAAAJCIZhsAAAAAJKLZBgAAAACJaLYBAAAAQCKabQAAAACQiGYbAAAAACTS2NEFdAUt7x8TZo8c9qOcK3skr+WXLw/Lza9/Z3wEb9uSpWFWd8DehWsCAv22C6Njdp5XkSmfmVgXZjs92BxmbY/G+wPUsuHfmR9mM393YqEx617fEmZjHr+30Jjb4sWBA8Ls9rn9wuyIPq8Umu/wh94TZjv89ZEwKxWaDbqPtnL+KtjSN/5q1yt1MUAh7//OrDA7eYc1YXbExE/kjtt71lOFayrig5+8KczG94p3nI9+c2aY7XzXnG2qqbPxZBsAAAAAJKLZBgAAAACJaLYBAAAAQCKabQAAAACQiGYbAAAAACSi2QYAAAAAicTnQ/P/apqxKvmYv391tzD7wdK3hNmuny/njtu2ZFmhel7Yd4dC1wGxtuWPh9lX//ieMDvuxAsLz/nI+84PswNe+lSYDXt0aeE5oTsrb3k9zNqWLK9iJZXz9Luaw2zfnjfkXNmr0HxPPrlzmG2/YWWhMYEse2ZijzAbdnMVCwFC617fKcxK2eowa+1TV4Fq8pUOOyDMjt3+gjDbUu4TZq29q/97dBRPtgEAAABAIpptAAAAAJCIZhsAAAAAJKLZBgAAAACJaLYBAAAAQCKabQAAAACQSGNHF9AlfDQ+2n6vT54RZsNuawuz7R55KswGrl4aZvGI22bDrrVzBC90Bnt8dm4cnli9OoDa8OwnpobZ2Pc/Fma7NsT3QEWN+/zjYVap+xzoTMpbtoTZ0i2bwqy5R+/ccTfu/nrhmoB0lp1/UJhdN+CCMLv4xeYw22luS+6crVsv619q2GnHMHvus6+F2ZDG+P7grCenhdmuly0Is3KYdE2ebAMAAACARDTbAAAAACARzTYAAAAASESzDQAAAAAS0WwDAAAAgEQ02wAAAAAgkcaOLqAraFseH1E/+qw4y1P0aN5K2TL5lY4uAfi/9ahrCLMt3e1MbKDdnjl9Wm7+wU/cFGbv3+F7YdavvmfhmiLffHZCmJU3v558PuhK2p5+JszOXPGeMLtl7A2VKAcooGHP0WH2i+kXh9mG8pYw+8NX3hpmfdbc177C3qBlF+0eZg9P+GmY3b6xXzzm5M3bVFN34ck2AAAAAEhEsw0AAAAAEtFsAwAAAIBENNsAAAAAIBHNNgAAAABIRLMNAAAAABJp7OgC+N+e+Nq0MGvtW86/uC4ny7n0XWPm5I8bOH3tv4VZn1sWFikFat6WcluYlbJSFSsBGvbeM8yWntw/zA475OHktcwadkFunr8/9Cw05/ItrWH2novPDrPh1z0dZqVXVhSqBQCqqXzw+DB772WzwmxSr/hefuwtnwqz5uvva1ddb9Sqc6aG2fw3/SDnyrhd9IWffTjMmrLZ7Smr2/NkGwAAAAAkotkGAAAAAIlotgEAAABAIpptAAAAAJCIZhsAAAAAJKLZBgAAAACJxGe50i4NO+wQZpsOHBNmPb70dJg9OPaCwvX0qGsIsy3l+AjiPH/d2DfM1n5seJiVWxcXmg8Aqql88Pgw+9AV14XZsds9V4Fq8lT/f6RnLn9PmDV9d3aYFbvjALbF9jtv6OgSoFOq69EzzNadPinM5n82/l6e/707/rx+1/iFYXbjd6eG2eivPxBmWZZl9bsNCrMZx8wNs4asLszGz/5wmA3/TnwPwN95sg0AAAAAEtFsAwAAAIBENNsAAAAAIBHNNgAAAABIRLMNAAAAABLRbAMAAACARDTbAAAAACCRxo4uoLOo69UrzF4/bN8wO+uiX4TZm/vcEWZPt20Os79u7B9mX1t6bJhlWZZdvfeVYTakMf4d8/Su3xJmK9+9U5iNWtI7zEqbNhWqBQCqqSErh1l9lf9n2aOuITffEpda2C3jrguzQ0/6ZJjt+Ku56YsBcl074adhdkZ2cBUrgc7lqVMnhdl9n/1RmJVyxsz7zL3q5aYw++/d7o2z98fZl484KKeaLDtyx5vD7M19Xg2zezfH39mHn/BQ7pzk82QbAAAAACSi2QYAAAAAiWi2AQAAAEAimm0AAAAAkIhmGwAAAAAkotkGAAAAAIk0dnQB1VTfOz7Wdv17Dgizu/77/ELz7X31GWE29K9tYdbrT/PCbMDg+NjeLMuyq2+dGGZnD3g499rIQb22hNmDH4r/NlPXnBlmu171QJiVNmxoX2HQTfWoawizvGPGt2aHac8Uvxi6sbp7FoXZZe88Osy++KEBYTb81tfDrGFja7vqSmnZKT3C7LGjL65iJUCeNXcPi8Ox1asDuppnT50aZrO/8MMwe6UUf9d9dMt2YfaVz348zHqvj+8B7vjvVWF2xcg/h9l/73ZvmGVZltXnPEdVyrluUs+41rOWLw6zHx33rni+B+Lraokn2wAAAAAgEc02AAAAAEhEsw0AAAAAEtFsAwAAAIBENNsAAAAAIBHNNgAAAABIpLGjC0itrlevMHvsB/vF2bHnF5rv2CXvDLPmc1eGWdvTz4RZ47ChYbb/jU/k1vO5AY+G2Uul+Fjfg649O8wGj41rvWPf34bZnP+I/6bvOXF6mD13/r5h1nt9fDRznoa/LSx0HXSELeW2MCvlHt6d7879rw6zGVNOiS+c+2DhOaGra3t0aZiN+nwVC9lG45btEodHV68OIN/2a8qFr+1XF1/bsFdzmOXtc9BV7PWBxWF242u7htl/X3pimA3+/uww65vd277C/sn6s+OexFkXHBpm5w25q9B8W9NQVxdmn3vouDAb8kDcd+DvPNkGAAAAAIlotgEAAABAIpptAAAAAJCIZhsAAAAAJKLZBgAAAACJaLYBAAAAQCKNHV1AEXWNcdlLfrh/mD0248IwW9u6Ocxm/OTzYTby8hVh1vr0M2G25YiJYbbPd+8Ps/8ctCDMsizLrnh5RJj94ivvCLPRf5gbZg0DB4TZvx15Rpi99p6Xwuy6A34aZkPP7xVmeWa9Ftd5afOoQmNCRxj7l4+E2aOHX1qROZd+rGeYNcfbA9BFPP2u0R1dAtAO9a3Fr22oqwuzUp8exQeGLmDBrXuF2fO/GRhmg5fMrkQ5oY279g6zM3b5S86V+Wt4yjdOD7OBD7y2tbL+pWHLW8KsrdCItcWTbQAAAACQiGYbAAAAACSi2QYAAAAAiWi2AQAAAEAimm0AAAAAkIhmGwAAAAAk0tjRBRSx5nMHhtljM34UZk+2bg6zE77zuTAbef3KMHv+8N3DrPz+fmH2+33iOndp6BVme//mjDDLsixrvvS5MOu75N7cayNtz60Psx2uzsviMY8/7fNhtuvxq9tV1/9y9k454SPFxoQO0Gtpnzg8vHp1QFdT1yv+/HzxhAPCrP8N8WdE6ZVXtqmmalp39rQwu+HM/8m5Mv67AdXV/8o5YXbJ50fkXnvqjvE99LKzeobZ6PdvvS7o7IZ/fXaYtVWxjizLsoZddgmztce1htnoHvHn8a9eGZw758CfxHtHUdX+u3U3nmwDAAAAgEQ02wAAAAAgEc02AAAAAEhEsw0AAAAAEtFsAwAAAIBENNsAAAAAIJHGji6giIs/elGh63rXxdk7Tv0/YdZ05gth9sEd/lioliyLj/Xd+9dnhtnoL83LHbWtNT5KuDMZdFF8NHO52MubZVlL0QuhUxn2zXh9XH1SU+61J/VbV2jOx4/+WZi9bf8Tw6z0wOJC80FRm95xYJjt+NknwuzO0ReE2cx58Xs8W/JKu+pKpXHwbmHWcvyo3Gt/e8b3wmxIY3zfkefpts1h1mNjudCYQDHfm3tUbn70W34YZs0fXxpmpaIFAf/SsrNHh9nit5wfZnM29wiz3804dCuzrthaWVSZJ9sAAAAAIBHNNgAAAABIRLMNAAAAABLRbAMAAACARDTbAAAAACARzTYAAAAASKSxowso4v+8OjbMDur1UJjt3BAfe//lgYsK1TL9sXeF2RNzhobZqN+/FGajH1kQZuXW1vYVBnRLVz4xLTc/ce9rCo27pVzoMqi6o751Z5idPeDhQmM+9uUd4vDVgwqNWdR7p80Js+sH/Sn32lLWo9CcH1x1VJgtv2LPMBvwh7hWoPrasrowK23cVMVKoPtr2Ks5zL458zdh1laOb7pPvvHUMBu9dG77CqPT8GQbAAAAACSi2QYAAAAAiWi2AQAAAEAimm0AAAAAkIhmGwAAAAAkotkGAAAAAIk0dnQBRcx+85AwO+ikw8Pspf1fD7PGZ3uEWfMlLfF1Tz0TZiM3rQmzUpgAxDZfuVv+D5xbnTqgO1l8xE86uoR2yv8f6ZxNvcLso/d+IMxGf3RZmA14bc7WywI6hT0a+4TZ+pMPDLMBl1nn8Ea9+w9/C7OZ28c9gglzTw6z0Z+euy0l0cl4sg0AAAAAEtFsAwAAAIBENNsAAAAAIBHNNgAAAABIRLMNAAAAABLRbAMAAACARBo7uoAi2tY/H2a7nj87zgrO11rwOoDU+i+K978sy7ILX9gzzD7Zf0nqcqDq/nLmwWF21WkHhtkDB19eiXIK+eXLw8Js3ZadwuzyhfHvnmVZNvqnbWE26p5FYVbKHRXoLK44LH8fe6G0McwGPvhqmJULVwS161s3HBdmJ77//DDrc9MOlSiHTsiTbQAAAACQiGYbAAAAACSi2QYAAAAAiWi2AQAAAEAimm0AAAAAkIhmGwAAAAAk0tjRBQDQfm2PLs3Nb90nPk781mxywVkXF7wO0mv428Iw2/2+vmE28cxPhdnPP/7DMNunZ12YHf7Qe8Lspb/tFmYjftsSZq2Prw6zMdmCMAO6v88tPj43P37E/WFW/9rmMGsrXBHUrlFfmBNmM74Q33MPyOLr6F482QYAAAAAiWi2AQAAAEAimm0AAAAAkIhmGwAAAAAkotkGAAAAAIlotgEAAABAIo0dXQAAQAqlDRvCrOk7s8Psy985sNB822crC2WthWYDat3O05fm5n/JtstJ868FIC1PtgEAAABAIpptAAAAAJCIZhsAAAAAJKLZBgAAAACJaLYBAAAAQCKabQAAAACQiGYbAAAAACSi2QYAAAAAiWi2AQAAAEAimm0AAAAAkIhmGwAAAAAkotkGAAAAAIlotgEAAABAIpptAAAAAJCIZhsAAAAAJKLZBgAAAACJaLYBAAAAQCKabQAAAACQiGYbAAAAACSi2QYAAAAAidSVy+WOrgEAAAAAugVPtgEAAABAIpptAAAAAJCIZhsAAAAAJKLZBgAAAACJaLYBAAAAQCKabQAAAACQiGYbAAAAACSi2QYAAAAAiWi2AQAAAEAimm0AAAAAkIhmGwAAAAAkotkGAAAAAIlotgEAAABAIpptAAAAAJCIZhsAAAAAJKLZBgAAAACJaLYBAAAAQCKabQAAAACQiGYbAAAAACSi2QYAAAAAiWi2AQAAAEAimm0AAAAAkIhmGwAAAAAkotkGAAAAAIlotgEAAABAIpptAAAAAJCIZhsAAAAAJKLZBgAAAACJaLYBAAAAQCKabQAAAACQiGYbAAAAACSi2QYAAAAAiWi2AQAAAEAimm0AAAAAkIhmGwAAAAAkotkGAAAAAIlotgEAAABAIpptAAAAAJCIZhsAAAAAJNLY3h88sv6EciUL+WezWhaE2fSmiVWspDJeP3pymPW8ZV4VK/m7l2/eI8x2eNuK9BPWN8RZqS2MXrppdJjteMzyMFt/ytQwG3DZnLiWHLeVrqkrdGEXVYk94ILV94TZiMaeYTajKV4/XcXK78bvyVFfKPae3BbjFsQfB4sntlaxksrI+0y5eUO/MLtwTHOY1dIeUO17gO7uiIdfCbPb94nfj1tzY0t8/1B037ypZWGYHdM0odCY1bby1+PDbNT7FhUas5bWf5ZVfw+o3267MCu99loVK6mMjcceGGZ9brivipXUhkp8r6ylPaAS6//120aE2TN/bQqzof89O3Up5KjEfUW1vX7UpDDreev8QmO2d/17sg0AAAAAEtFsAwAAAIBENNsAAAAAIBHNNgAAAABIRLMNAAAAABLRbAMAAACAROrK5fad5FvtI78roRLHPnem+Wrdmq9OC7Nh56Q/JrqWjvzOsvw9YOR9fcLrVh24sSL1FLFp+oFh1nvWfVWsxP7QVX7/vDp7DV5ZM3tAZ7oHWPP7fcJs2PEPV7GS4m5qWRhmxzRNqGIlHaMSv/9zH58aZgN/MqfQmHncA9ARrlubf68yc2h8n9OZNOzVHGZtjy4tNGafO3cNs42HPV1ozDy1tAd0h/X/+lGTwqznrfOrWAnPz4rX/87Ti63/G1vmhdl7lk8Ps6J7Q3vXvyfbAAAAACARzTYAAAAASESzDQAAAAAS0WwDAAAAgEQ02wAAAAAgEc02AAAAAEikrlxu30m+3eHI3zzrT4mPjB9wWfoj4zud+oY4K7WF0YaZB4VZn2c2h9mK43qH2ejPzA2zNV+ZFmbDvjU7zCqhlo78zrLie8CL/x6vrZ1+0XnWVt6R0TOaJlexkq7lk8viI7ovHBMf7V1U6+ETw6zxLwuSz1e6Y1iY3fHmH9TMHlB0/R/64KYwu2u/+HOg6gp+BhJbetmkMGs+ZX4VK6kM9wDdy3Vr7wuzmUMPrGIlHaNxaFOYta5tKTTmrJb4M3l6U/xZ3lVei1raA4qu/4ZdB4VZ29PPFK4ntWVXTQizMR9YWMVKuo+zli8Os/NGjwuzjcfGa7z+9fht2Ovm+HtcUXl7WK/BK9u1/j3ZBgAAAACJaLYBAAAAQCKabQAAAACQiGYbAAAAACSi2QYAAAAAiWi2AQAAAEAideVy+07y7e5HftO97Tm/R5h9f/DcMDu95ZAw++mkq2rmyO8sy7KPzv9AuAesOnBjNUvJSoceEGb1d91fxUrI87u1c8Ls3UOnFhpz9Tfi60Z8LZ7vxpb4SPC3nfyJMOvx5/lhdlvpmprZA9wDUIvqJu0TZn++7z9rZv1nWefaA65be1+YzRx6YBUroRLyXt+3nXp6mPWeFV9XCe4BOkbe/dyMpslVrIRKmPnos2F2/QFDw6y8eXOYzWpZEGbTmya2r7B/0t7178k2AAAAAEhEsw0AAAAAEtFsAwAAAIBENNsAAAAAIBHNNgAAAABIRLMNAAAAABKpK5fbd5LvxI/8IPzBnS+fE15XiaNWi1p6cXwcePMn0h8X3eNvg8Ns3A5PhdmDEzrN6codojO9Z/LU0pHfWZZlIy/4fvjGXHL8heF1pawUZrV8RPdzf2wOs4HvWFrFSv6u7c0TwqzhrwvDrPXweE02/iVey3kaR42M51u5qtCYlVBLe8CR9SfU9gdTAZvfFu9vG05/Mcz6v31ZBaohtVpa/1lWfA+4bm18fz1zaHxfXsv8zbqGWtoD3jbirHD9t65ZW81SoFNo7/r3ZBsAAAAAJKLZBgAAAACJaLYBAAAAQCKabQAAAACQiGYbAAAAACSi2QYAAAAAidSVy+07ybvokd9F1U3aJ8zK8x8uNOaslgVhNr1pYqExqV21dOR3llV/D6iEG1vmhdmMpslVrITuoJb2gO6w/vNcu3ZumB187mfCbLcfza5EORWx8tfjw2zU+xYln++C1feE2RkjDk4+X7XV0vrPsurvAdetvS/MZg49MPl8DXvvGWZtjyxJPl8t6O7fu2ppD+ju9wC+H6TX3f+m7V3/nmwDAAAAgEQ02wAAAAAgEc02AAAAAEhEsw0AAAAAEtFsAwAAAIBENNsAAAAAIJHGji4gUp7/cJj9bu2cMHv30Klh1h2Omd6aznTMdreopb6hAtV0P9U+3rm+b98wK23YUNVaqu25j8V7XJZl2cBL4/2x2jrTsd+daT9i27zwpzFh1v/ty8Jsz/k9wuy4oVPCbLdsdvsK6+RGvW9RVec7Y8TBVZ2vEvLea1TWzKEHFrruurX3FRqz7ZElheYjVu3P1qKvPR3jqbOmhdlu5xX83J2yXxh98he/D7Pu8P2gs6mv8jNdQ+b2C7NLh/0tzCq9T3myDQAAAAAS0WwDAAAAgEQ02wAAAAAgEc02AAAAAEhEsw0AAAAAEtFsAwAAAIBEGju6gCLePXRqoeue/3B83c6XzylaTlV9ctnS3Lzo8bVPnxkfvzzvCxcUmq9oLbNaFiQfs/CxvqW2YtfVmGofmV3asKGq8z33sXjvGHhpsb2j6Pv8wi/+OHfc/7y02Ht9/Ufj3/Ge/zo/zPJe+6Lvixtb5iUf84XSpkLX0fl8b9w1YfbZP50QZn96bPswG53dv001vVHHLX4mzK4dN6iKldSGQx+M1/9d+/UOs/5vXxYPWtqWisiyLDt5yeowu2LPEYXGnDn0wDC7YPU9YXbGiIMLzVdtlz1xd25+yvBDCo173dr7wizvb1oJH1iyJsyu2nNYmFW7TrbNbufNLnTdCx+K71f7Xxnfk+/dM/7czbLmQrV0F8+eGv9NL/j8hWH2jVETwqzwd++CnpzySpjNWdlQxUr+kSfbAAAAACARzTYAAAAASESzDQAAAAAS0WwDAAAAgEQ02wAAAAAgEc02AAAAAEikrlwut+sHj6w/oX0/+E92vHtAmL10yPoiQ1Zd4+Ddwqx13VPJ52vYaccwa3vxpdxr6/v1C7Phd7SG2cPPDw6z7Y9emTtnd7bL7J3C7NdTflpXvUo6XtE9YOlF8VHszafFR813Jj9YFR8l/pmR8XHZReW9756d9mLhcb++ckGYHdCrFGYzmiYXnrOI5z4W/00HXhq/FpWw/qNxLQt/8pma2QO2rNsjXP/HNMVHv39gyZowu2rPYdtWFF3WqnPidTXyq9Vd40XdVrqmZtZ/lhW/B6BruG5tfD/Wo64hzKY3TaxEOV1CLe0Beev/xpZ54XWVuH8ccE//MFt/8AvJ56u2De86KMx2WPBk7rXXz74+zN7+7lPCrO6eRVsrK6nHvx3fA+z+pe51D+DJNgAAAABIRLMNAAAAABLRbAMAAACARDTbAAAAACARzTYAAAAASESzDQAAAAASqSuX23eSd96Rv6t+u1943cj3PBhmSy8+MMyaPxEfQd1VzGpZEGZFj8rOG3NbxuVfy/t79xq8smaO/M6y/D2gqEocF16JdVdU0d9vz/k9wuzcwbNz56zEMet5fvrE3WH20eGHVLGS4trePCHMbv7lT8OslvaAQ2aeG67/Pjd0ns/rm1oWhtkxTfHr3F28dvxBYbbd7+8Ns3XXjwuz1taGMNv1Z33CrOct8f7XHdxWuqZm1n+WVeYeoLu7bm28N84cGn8HyrP00vzP+OaPde91t8e83mG2YvKmKlZSW3tA3vqv69EzvO6GVfeEWSXuVxuHDQ2z1jVrk89XCZX4btRdvPLeKWHW7zdzq1hJ+9e/J9sAAAAAIBHNNgAAAABIRLMNAAAAABLRbAMAAACARDTbAAAAACARzTYAAAAASKSuXG7fSd7VPvL7d2vnhFmPLD6GvuhR2pVQN2mfMCvPf7jQmMt+fFBuPub0e+Nrr5oQX/eBhYXq2XN+jzBbMmlLoTG7ilo68jvLsuy9cz4W7gHrD34hvO6ps6aF2W7nzQ6zvKOvv/ZMfPT1ogPCqFtY+tP8Y7+bPxr/3Sphj3m9w2zF5E1VrKT6amkPOHrPL4Tr/1d/+WV43XuHxes/z9JL4s/y5lPvKzRmtdVN3jfMyvMeqmIlHWPlr8eH2aj3LapaHZVSS+s/y7Lse48eFe4Bt+6zQ3jddWvj9Vr0nn35D6aE2e9mnh9mX949/XeESvx+lfLMafF+POii+H6sO6jE61RLe8DmdaPC9X/s3oeH1133yO2F5pvRFN/rrv56/D5+4CM/KjRmUY27jwiz1sdXJ5+vs8n7rlaJv3dRVz5xd5h9aPghhcZs7/r3ZBsAAAAAJKLZBgAAAACJaLYBAAAAQCKabQAAAACQiGYbAAAAACSi2QYAAAAAidSVy+FJvv/gyPoT2veDb0B3OC536U/iY22bPx4fh1sps1oWhNn0pomFxnzbIy+G2c1771RozGUXHBRmY864N8wev3r/MNv9xAcK1VJULR35nWVZNvKq74R7wJgPxe+7hv79w2zLfiPDrP7O+9tXWAf7+sr4dz+gVynMDv3KmWHW1jOeb+Clc3LrWXZ+zto6M15bvHG1tAdU4h6gu2vYc3SYNV76aphtPuypwnPe1LIwzI5pmlB4XP63Wlr/WZZlY//w9XAPGHb8w4XGvPXJRWF21JDxhcZc8f0pYbbH2XMLjVkJT31qWpjt9qPZVaxk6yrxfS3v+8r9r8f3Tv+xe/y9q9pqaQ/oKvcAlfge3DBwQJg99r2RYZb33agSdXY2L90U3wPteMzyKlZSGe1d/55sAwAAAIBENNsAAAAAIBHNNgAAAABIRLMNAAAAABLRbAMAAACARDTbAAAAACCRunK5fSf5dqYjf/OOy81T9Cjdi1bfHWanjTgkzCpxrG/jyOG5eeuqJ6paTy2rpSO/syx/D3h+VnN43c7TlyavpXTHsDCrf8ua5PMV9YNVc8LsMyOnFhrzxpZ5ufmMpsmFxv3Y0pVhdmnzqEJj1k3cO8zKCx4pNGZnUkt7QN76X/G9KeF1e3x2bkXqoWt76qxpYbbbebOrWElxtbT+s6xzfQ9o2Cu+52h7NP09x9ovxe/Xod+u7vu1dNgBuXn9nfdXqRJqaQ/oTOv/lffG9xz9fpP+nqN+/F5hVlr0aJhdktM/ODWnf9BdLLvgoDAbc8a9VaykMtq7/j3ZBgAAAACJaLYBAAAAQCKabQAAAACQiGYbAAAAACSi2QYAAAAAiWi2AQAAAEAideVy+07yHXHJueEPjvvaivC6tufWv/GqOplZLQvCbNpXTw+zna+YE2ZTHtgSZnP379G+wjqBvL/N9KaJVawkX16defJ+h1o68jvL8o/9zjuKPu8Y+mduGBtmg459rJ2VVV59375hVtqwodCYXWXtZFmW7TJ7pzD7ypCbwuzTI6dVoJr0bmyZF2YzmiaHWS3tAXnrv6u4qWVhmB3TNCH5dd1F6dCc/f2ueH/v7mpp/WdZlp1838nhHrB2yqvVLIVO5Kmz4s/53c6bXcVKqq+W9oC8e4Ci74Hc++DjTo6LmftgnHUReffVz057sWp1tEfDuDFh1rZ4WRUrqYxed+4WZpsPeyrM2rv+PdkGAAAAAIlotgEAAABAIpptAAAAAJCIZhsAAAAAJKLZBgAAAACJaLYBAAAAQCJ15XJ4ku8/yDvytxJyjwNumljFSipjw8yDwqzvdfdWsZK/63fXwDB75dDnCo1Z9DW87Im7w+yU4YcUqqUSaunI7yyr/h5Qy5ZeEa+P5pPjdVUpN7bMC7MZTZPD7LAHN4bZ2QMeLlRL3nxF5f1+4+74eJitev+XamYPyFv/j39nanjd7l+cU2i+huY9wqxt6YpCYxJ75vRpYTbox7OrWEn1td4+PMwaj3gizNwD0J2s+f0+YTbs+GKf10V9YMmaMLtqz2FVrCRfLe0B1V7/X1m5KMy+NWp81eqoFXn3wUc/enyY9TxydZg1DBwQZtc9cEuYVeI+P8/rR00Ks563zg+z9q5/T7YBAAAAQCKabQAAAACQiGYbAAAAACSi2QYAAAAAiWi2AQAAAEAimm0AAAAAkIhmGwAAAAAkUlcul9v1g0fWn9C+H0xk9TemhtmIr82pYiVZ1jh4tzAr79QvzNoWLwuzS1bfHWb96uvC7KRhB4fZtpjVsiDMpjdNrMicXd1tpWviF6obKroHFH1vNew5OszaliwvUkphDc17hFnb0hWFxqzv3TvMSps2FRqT6qqlPaAS9wA3tSwMs2OaJnSaMSuh7i9Ncfj5/mFUnv9w7rhXPhHfW3xo+CFh9s3H54XZf+w+OXfOWlVL6z/Lqv89oLvfl2489sAw63PDfVWs5O8aRw4Ps9ZVT1Sxkuq7bm389545NH6damkPqPb6p7qKroGu4sTHngyzq8cOKTRme9e/J9sAAAAAIBHNNgAAAABIRLMNAAAAABLRbAMAAACARDTbAAAAACARzTYAAAAASKSxIyfPO9b77cfvH2YNo3cPs7blj29TTf9K67qnwuzaedeH2XFDp4TZ0MY+YdYRR5oXnbNx1Mgwa125Ksxe+ODUMOv/8zmFasl7P928oV+YXTimudB8tE/ee2vmo8+G2Sk7xq/n4i1bwuxzI+N1V1Tb0hVh9uR1e4XZkJmPhtmNK+4Js47YAyohb02+9ZRTw6znLfMKzffMDWPDbNCxjxUak8o5pmlClxgzz00tC8Msr5by4S05o+Zl+e7a1FTouv/YfXKh6y5afXeYnTbikDC79clFYXbUkPGFannhT2PCrP/blxUak/bJ2+vzPs9+tSb+HPzrxp2Sz9eZ9Lnhvo4u4R+0rnqio0tol+vWxn+3mUMPTH4dHaOhf/8wa3vhhSpWUhk3tsT3uTOain0eb4tKrIHO9Ds+siHv3qhc0bk92QYAAAAAiWi2AQAAAEAimm0AAAAAkIhmGwAAAAAkotkGAAAAAIlotgEAAABAInXlcvuOOz2y/oTwB6t9BHddY2OYlVtbw+zzKx4Kszf1fj3MKvE75P3N8nS6I83r6uKsne+tf3bm8sfC7PzRYwuNmWfZj6aE2ZhPzQ2z20rX5Pzy3U/eHtAdnLV8cZidN3pc8vny9oCxv/tkmI0+K35PVkre8d3Lt8R77mdGTq1EOZ1GLe0Beeu/rlev8Lry5s0VqYe0lv/ygDDb8+yWMGt7+pkwu6llYZgd0zShfYV1YrW0/rOs+D1Atb8jDLinf5itP/iF5PMVlfd3Ofz008Ks73X3VqIcCqilPaDo+n/hg/F9YP+fzylcTyTvfrU+5xmjTvf9uovI+3vPaJpcxUqKK/o7tHf9e7INAAAAABLRbAMAAACARDTbAAAAACARzTYAAAAASESzDQAAAAAS0WwDAAAAgETqyuX2neR75dKp4Q/+auzQQpPnHXt9yYuj4uv2jo/1ppjNfx4ZZr3euqpqdWRZlj3zyWlhNujC2cnnq/SRv91F0WO/8+TtAf9nU88w+5899k1dSrbHvN5htmLypuTzdTaVOL67310Dw+yVQ5+LL7wj5zPlLWsL1VLUK++dEmZzf312zewBlVj/L/xpTJj1f/uy1NN1GSu+H7/n9jh7bhUr+bvXj47Xf89b4n2jq2j5QnzP0fTd+J7DPUBl5d0fTG+aWMVKasOKc6fG4ZD4HmiPk+6vQDVdQy3tAdVe/3QeSy85MMyaT70v+XwNAweEWdtz65PPV1R7178n2wAAAAAgEc02AAAAAEhEsw0AAAAAEtFsAwAAAIBENNsAAAAAIBHNNgAAAABIpK5cbt9JvpU48jfvWO88tXzk99b+Znl/m6fPjI+33/X8+Hj7bqEu53Tedq6Bf1ZLR35nWWX2gIbmPcLsur/+JsxmNE1OXUrNW/P7fcJs2PEPV7GSrqOW9oBKrP+i1l0/LswGv3NxFSupvp8+cXdu/tHhhySfc+Wvx4fZqPctSj7fU2fF9yq7nVfsXqV0x7Awq3/LmkJj1tL6z7IK3QPsvWeYtT2yJPV03cLQudvn5munvFqlSrqWhl12CbO2Z58tNGYt7QHVvgc44uFXwuz2ffpVsZJ8y34+IczGfHBhoTFvbJlXtJzc70dLL5sUZs2nzC88Z3JT9ouzuQ8WGvKFP40Js/5vX1ZozPauf0+2AQAAAEAimm0AAAAAkIhmGwAAAAAkotkGAAAAAIlotgEAAABAIpptAAAAAJBIY7t/cPBuYdbwm/i6zYc9FWbTmya2d/okGnbaMcxWXzY0zIYe90glyilkW/5mQ26OX4u2nOtmtSxIXs+zN8bHve8yo9hx70Xr/MGqOWH2mZFTC9VSa0qHHRBm9XfeH2ZtS1eEWd7x1UW98t4pYdbvN3OTz1fUrU8uCrOjhozPvXbjOw8Msz7X3xdmw45/OMw2vy1+LXrdXPyI8tQ+uWxpmL2tb3yMfLU/i9g2g9+5OPmYDQMHhFnbc+sLjbnu+nFhlvc71E3eN8yaGhYWqiXLsuzatfEed9zQeG+se6JP4Tkjrx1/UJjtdt7s5PPVv2VN8jHZdm2PFLvfq4RK3OtWwtopr1Z9zkr8bbblPqeItmefTT4mf9c4LP4O3bpmbaExb9+nX9FyQvX94jGvf+wvYZb3fWTMB4t/JheZb1s0nzI/zNb8x7QwG/bN9J/JeWZde0WY5e03+fvUNpW0TTzZBgAAAACJaLYBAAAAQCKabQAAAACQiGYbAAAAACSi2QYAAAAAiWi2AQAAAEAije39wdZ1T8XZYUlqqbgbHomP9d37/3y40JhFj8M+9MFNYXbXfr3D7PmTp+bWs/MVc8KsbdnKMHvqU/GRv5U4LneXGemPey96/PhnRub/Tdm6+jvvr+58ffuGWWnDhjB7fq+6MPuv5Y+E2fdH7x1mRfeA124ZFWZHDQmjbOkV+e/z5pPvy82L6HXzvORjNvTvH2ZtL7wQZi+9f0qYXTgmnm+HFQ+2qy7euOc/HO+hO18efyZ1Jm3PrU8+5uB3Lg6zzW+bHGZ56+2YpgmF6zluaLx28uz+xfSv4Xa/vzf5mHlabx8eZo1HPFHFSuistpTbCl1X9B6gK6nE73HUkPGFrnv8N/uF2e7v9TnfEVrXrK3qfCu+H3+W7XH23DC7/rG4D3Dcshk5M65rT1lVceuTi3Lztx80PczyXqdh35xdtKTkiu43edf99+Pxd6Mv735gofnay5NtAAAAAJCIZhsAAAAAJKLZBgAAAACJaLYBAAAAQCKabQAAAACQiGYbAAAAACRSVy6X2/WDR9af0L4f5P/16i2jwmz7o1eGWVc6RrzH3waH2ZZ/q+5RyZX4u+WN2WvwyrpCg3ZReXtA3lHUece7r/zO1DAb9cU5Yfbcx+LrBl4aX1cJN7bMC7MZTZMLjbnu+nFhNvidiwuNWSl5a6SUlcKs6N8mT95rMf6nnwqz3gc8H2aDjn0szG4rXVMze0De+r+pZWF43TFNE5LX0nr78DBrPOKJ5PORXrVfw0rcH9TS+s+yynwP6Er3u/DPamkPyFv/q86J78n3+OHSMHvyxD3DbNcLZrezsu7n2VPjv+cul1T3O87WDLinf5itP/iFKlaS/x2gPuf5skrfA3iyDQAAAAAS0WwDAAAAgEQ02wAAAAAgEc02AAAAAEhEsw0AAAAAEtFsAwAAAIBE6srl9p3knXfkb97R3S+VNoXZScMObtfctN8Fq+8JszNGxH/vhl0HhVnb089sU00ptf3bhDBr+NvCQmPmvX/z9Bq8smaO/M6y/D0g77jlPDOaJheupyvIe2/lHTXdOGxomLWuWbtNNaX23B+bw2zgO+Ij3zuTou/fPoNX1cwekLf+a9lNLfHnzqRvnx5mg348uxLlZBetvjvMThtxSPL5nrxurzAbMvPR5PNVW97r22PwippZ/1lWfA949YSDwmz7a+4tXE93VvTeIcuy7Lq194XZzKEHhtkHlqwJs6v2HBZm9dttF2al114Ls+7gttI1NbMHFO0D/PCF+B7x9n36bVtR3VTePenWvjdty97BG9Pe9e/JNgAAAABIRLMNAAAAABLRbAMAAACARDTbAAAAACARzTYAAAAASESzDQAAAAASaaz0BCcNO7jSU3Sox6/eP8x2P/GBQmNetPruMDtzyvG5154xotjf+4aFNxe6Lu8Y4RXfnxJme5w9t9B8DX9bGGYnPbY2zH41dmiYFT0K+bZSocu6rDW/3yfMZr22MswubR5ViXJC23Jkdur5phec7w9zrwuzSvwOWZZlDTvsEGYfWxCvu4vHxGN2ptcib76itdTaHlBE48jhYda66okqVlIZxzRNCLNB2ewqVvJ3p404JMzWXT8uzAa/c3Gh+eru3qnQdV1F3utr/bdPnb/TG/bI662Fr5059MBC112157BC15Vee63QdZWwYeZBYdb3unsLjVk67ICi5dSMd045Nszy7mdvzypzP1tN1b7P3Zpj9z48J32p0JjLroo/B78y+aYw+9243QrN1914sg0AAAAAEtFsAwAAAIBENNsAAAAAIBHNNgAAAABIRLMNAAAAABLRbAMAAACAROrK5XK7fvDI+hPa94MdrH78XmFWWvRonB0yPh7z7kXbUFFas1oW5ObTmyZWqZLKaRwcHxXcuu6pKlaS77bSNXUdXUM1dZU9IE9nO6K7iB+smpObf2bk1ORz5v3d6nP+Z1N0PyoddkA83533FxqzEmppD5h085fD9d//7cvC69ZdPy7MBr9zcZidu2pumH1u5JQwy3Pog5vC7K79ehcaM+8zuej7//Fvx2t49y/lr//OpKvcV635/T5hNuz4h8OsltZ/lnWPe4A8Df37h1nbCy8UGvPVW0aF2fZHryw0Jp1HLe0B3WH9V/te9tYnF4XZUUPGFxqzcdjQ3Lx1zdpC4xbVHb5XFdXe9e/JNgAAAABIRLMNAAAAABLRbAMAAACARDTbAAAAACARzTYAAAAASESzDQAAAAASqSuX23eS75QTvx/+4J0/uDC8ruhxuZVQv/+4MCs9sLjQmCcvWR1mV+w5otCYeV4/alJu3vPW+cnn7A5mtSwIs6Lv0Vo68jvLih/7/dzHp4bZwJ/MKVxPao2jRoZZ68pVYfbcx3J+v0s7z+9X6+wB26bo+u8qWm8fHmaNRzxRxUq6j3XXx/dcg99Z7J6r7d8mhNkLzb3CLG8vvvXJRfF85VKY9Ri8ombWf5Zl2X889M5wD5i7f49qltKpVOKzha6hlu4Bjur3oXD9l157Lfl8DbsOCrO2p59JPp91nN7yH04Js9GfnltozMYRw8KsdfWaMLuxZV6YzWiaXKiW9q5/T7YBAAAAQCKabQAAAACQiGYbAAAAACSi2QYAAAAAiWi2AQAAAEAimm0AAAAAkEhduRye5PsPjqw/oX0/+E+W//KAMBv9/vuLDJnrteMOCrPtrr03zPKOfj9qyPhtqIh/ZZfZO4XZs9NeDLNKHAX9+Lenhtno7z4aZrc8/7OaOfI7y7Js47qR4R5Qn9O3r/aR2ZU43rnaNr8trrPXzfHv1xGK/r0bm4aEWWvLk9tUU0p1B+wdZn9e8F81swcUvQf4+soFYbZiS7yfX7VnfLx7JXxt5cIwe//fPhZmzR+eX2i+m1ri+Y5pmlBozM7m8e/En627f3FOFSvJsj3n9wizJS/tGl/4lrVhdFvpmppZ/1lWfA+otrOXPxJm3x8d7+cb33lgnPVvCLOdr6jue7nW1U3aJ8zK8x8Os1kt8WdR0fvUWtoDnmkZEq7/k4YdXM1Sqm7MvF5htmzy5uTz5d2PfGNU9e8PXj86vpfveUt1v5MU/c5x5vLHwuwb3zg5zHa6Kt7f27v+PdkGAAAAAIlotgEAAABAIpptAAAAAJCIZhsAAAAAJKLZBgAAAACJaLYBAAAAQCJ15XKXOMkbAAAAADo9T7YBAAAAQCKabQAAAACQiGYbAAAAACSi2QYAAAAAiWi2AQAAAEAimm0AAAAAkIhmGwAAAAAkotkGAAAAAIlotgEAAABAIpptAAAAAJCIZhsAAAAAJKLZBgAAAACJaLYBAAAAQCKabQAAAACQiGYbAAAAACSi2QYAAAAAiWi2AQAAAEAimm0AAAAAkIhmGwAAAAAkotkGAAAAAIlotgEAAABAIpptAAAAAJCIZhsAAAAAJKLZBgAAAACJaLYBAAAAQCKabQAAAACQiGYbAAAAACSi2QYAAAAAiWi2AQAAAEAimm0AAAAAkIhmGwAAAAAkotkGAAAAAIlotgEAAABAIpptAAAAAJCIZhsAAAAAJKLZBgAAAACJaLYBAAAAQCKabQAAAACQSGN7f/DI+hPKlSzkn9X16Blm5S2vV7GSfHV/aQqz8uEtVazk7+r3HxdmpQcWh9nSyyaF2cWH/SLMvveRk8Ks4W8Lw2zd9XGdg98Z15ln7Zemhdmw218Js2cmbR9mu1w8J8xuK11T177Kuodq7wHwr2z+88gw6/XWVVWrI8tqaw+o9vqfeH8pzBYc0PX/T3hjy7wwm9E0uYqV/F1ePfU5/5ed3jQxeS1D58afye8aOD/Mzh89Nszyfr93jj08zEqvxPcOtbT+s8w9AJ1D49D4e1fr2up+76qlPaAS679hz9FhtuQTA8Ns9Kfnpi6l6jrbPUCeWS0LwqwS9wCV+NsMmdsvzJ6cEn/O52nv+u/6d6wAAAAA0ElotgEAAABAIpptAAAAAJCIZhsAAAAAJKLZBgAAAACJaLYBAAAAQCKNHV1ApLzl9eRjrvj1+DDb432LCo1ZPjw+Zrphl13C7LWDdg+z3rPuC7OXboqPSc6yLNvxmMW5eaT5lPlhdl42LswasoWF5uv/s/gI3qJe3zE+lbo876Ew2yU+YTh79tSp21ISnczSyyeFWfOH4zVA59HrravCbM1Xp4XZsHNmV6AaKmXBAen/F1iJ4+TzrPmP+P2YZTkfPB2gEr9/nqKvxfnZ2ORjZtkrORndSePQpjBrXRvfz9N5eJ26j7Yly8Ns9KfjrDs4+8lDctLNYZLXW8iyLGt79tmCFcWmN01MPuZ3H783zGY0HVRozFXn5HxnnzKn0JgpeLINAAAAABLRbAMAAACARDTbAAAAACARzTYAAAAASESzDQAAAAAS0WwDAAAAgETqyuVyu37wyPoT2veDXdTXVi4Ms2+MmlDFSrqWG1vmhdlhD5wYZjseEx/p3DhsaJj9Ye51YTajaXKYVcJtpWvqqjphB+tMe0DD3nuG2U23/TbMjhoyvgLVdA95a7no2io65qyWBWFWiSPIi6qlPaAzrf+6iXuHWXnBI4XGrMT7vxI2vz2/ll5/in+Parv1yUVhVnQvbthzdJi1LYnvKyqhltZ/lnWuPYD0bmqJvwcd01Tse1DRMdd/ZGqYDfjZnEK1VEIt7QHWf+dQOvSA3Lz+rvsLjZv33bt1zdpCY+6zIH6m68Yl+4XZkWMeC7MVkzcVqqUS2rv+PdkGAAAAAIlotgEAAABAIpptAAAAAJCIZhsAAAAAJKLZBgAAAACJaLYBAAAAQCJ15XL7TvLtTEf+3tgSH20/o2lyFSvJ1zBg5zC77sFbw6wz/Q7bYsoDW8Js7v49Co3ZsOfoMGtbsjzM8t4z7zzq38Os9HB8/HAtHfmdZZ1rD6B7W/PVaWH2+o7x23CPz80Js7w9oD7n/07TmyaGWS3tAZ1p/Z+5PN6Xzx89toqV5DvswY1hdud+fapYScco+nn97CemhtkuF8drPM+yHx8UZotn/jjMZu5/dJjd8swlNbP+s6xz7QHUrs1vi78j9bo5/py/9clFYdZWLoXZMU0Twsw9AHQNjYN3C7PWdU+F2SWr7w6zPYata9f692QbAAAAACSi2QYAAAAAiWi2AQAAAEAimm0AAAAAkIhmGwAAAAAkotkGAAAAAInUlcvtO8m36JG/jSOHh1nrqieKDNntbTliYpj1uH1BFSshTy0d+Z1lWbZx3chwD5jRFB/Fzr+Wdwz9UUPGV62ObdWwV3OYtT26tIqVVF8t7QFF1/+NLfPCrLvvGy+dNCXM7vyfC8Ksu/9dtqarvGdqaf1nWZYd2fje+HtAqa2KlUDnUEt7QNE+AG/cxmMPDLM+N9xXxUrI097178k2AAAAAEhEsw0AAAAAEtFsAwAAAIBENNsAAAAAIBHNNgAAAABIRLMNAAAAABKpK5fbd5JvtY/8veyJu8PslOGHFBoz7zj5PJ3pqPmu5Kc5r+FHC76GnUktHfmdZY79hn9WS3tAd1j/efcA9Tn/ezz8jNPCrO8f7t2mmrqzWS0Lwmx608QqVlIZtbT+s6x77AGQUi3tAdVe/8uumhBmYz6wMPl87178VJj9btxuyeerBXn3XN2ht9Le9e/JNgAAAABIRLMNAAAAABLRbAMAAACARDTbAAAAACARzTYAAAAASESzDQAAAAASaUwxyJVP3B1mHxp+SKExT8m5bvkPp4TZsNvackbt+kfQ1vXqlZuXN2+uUiVb99GCr31RjYPjo5mvn/+nMJveNLES5VBJU/aLs7kPVq8Ocm3+88gw6/XWVcnnm9WyIMwOO/uTYdbvN3MLjcnfVft497z53nHCR3KujK/L+xzom93bnrI6hfq+fcOstGFDFSupvvK0/cPsj9f8LMzy3qP979l5m2oi3/OzmsNs5+lLq1gJlbD08klh1vzh+cnnu6llYZgd/b5Twqz+zvuT18K2GfOB+LXMuwc4edVbw+xbQ/8YZqeOqO531qJ2mb1Tbv7stBerUkd7VLu3Uu170fbyZBsAAAAAJKLZBgAAAACJaLYBAAAAQCKabQAAAACQiGYbAAAAACSi2QYAAAAAiTSmGORDw4sdl9uw66Awa3v6mTB79IQLwmzGp+OjXesL9haXXnxgmDV/4r5CY26aHo/Ze1Y85g0r784d99z1+4bZXfv1DrO843LzdORRuv9s1cmjwmx608QqVkLFzX0wjGa1LCg0pPdIbNc5O4TZ01NfDrNeb11VgWpiea9hv2xu8jFvKxUastvpTJ8DdbMf6OgS2mXyorYwmze+Icy29ln9/fX7hNmd+/UpNO6cTb3C7Nt77BdmRffUvFry3mt5r33R9+gLBz8fh9b/Ntt5+tKqzle6Y1iY1b9lTRUr6VoahzaFWevaljBr/vD8SpQTOqZpQpjVZ/dXsRI6yvqDXwizt373c2E2KpsTZk/857QwG/712e0rLJFnp71Y1fmyLMs2Hhv3LPrcUKwPkqfoPUBe9vQZ8Wu46wWVfQ092QYAAAAAiWi2AQAAAEAimm0AAAAAkIhmGwAAAAAkotkGAAAAAIlotgEAAABAInXlcrldP3jKvA+FP/jEQa+F1xU9vrUzaRiwc5i1rc85Fr6gdy9+Ksx+N2635PNlWZYtu3JimO365x5htsOv5xaab8sR8Xw3//ySMOtM75nbStfUdXQN1XRk/Qnt2yy6oVfeOyXM+v2m2BrIsy37ZnfYczuTWS0LwqzX4JU1swfs9aXzwvXf9N342PSXTorXzo6/Sr92KiHvPTC9Kf4s6whfWvFgmE3tvbnQmJXYN/L2qTydaQ9zD9BxGvZqDrO2R5dWsZLKuKllYZgd0zShYtfyv/1wVfz5tvfwlprZA/b6cs49wHfiv1H9/uPCrPTA4m0rqpvaNP3AMOs9674qVlI5XeW+6m2PvBhmnxl3W7vWvyfbAAAAACARzTYAAAAASESzDQAAAAAS0WwDAAAAgEQ02wAAAAAgEc02AAAAAEikrlxu30neeUd+P3XWtPC63c6LjwN+8nPxdUPOja/L092Py60fv1duXlr0aJUq6Rj7LIj7ww9PLCWf77AHN4bZV/eZVTNHfmdZ/h6Q56iHXw6zW/fZoXA91KblvzggzEb/+/3J53vlvVPCbO6vz66ZPSBv/df16BleVzow/sz60+8uC7OiR7/f2DIvzGY0TS40ZlGVqGXZhQfl5mM+eW+hcYtaevmkMGv+8PwqVlIZea9hn8Gramb9Z1nxe4BfrbknzE4adnDheqhNL3xoapj1v3JO8vkadx8RZjev+F7N7AFF13+1zWpZEGZF7yuqLe+e6g+P35177cyhcR+kEp45Le7lDLqoWC8nzw9XxWN+emRcSyXcVrqmXevfk20AAAAAkIhmGwAAAAAkotkGAAAAAIlotgEAAABAIpptAAAAAJCIZhsAAAAAJFJXLrfvJN+N60YmP/J3RtPkMMs7bv3CF/YMsyt+fnSYDTk3/RG0XUne3zTvteBfa++Rv91FVzn2u9pufXJRmB01ZHzV6miPNf8RH4s97JvF9seusq+s+WrO735Osd+9lvaAvHuA5Vtaw+tG92gMs7z3x0snTQmzO//ngjA7edVbw2z9wS+EWVHlafuH2ev9e4ZZrz/F6ybPuxc/lZv/btxuhcYtqqus/1+tuSfMThp2cKExa2n9Z5l7gMjqr8efLSP+s/t/77hgdby2zhhRbG11FbW0B+x/+nnh+p/7lR+F11X7c6Bhpx3DrO3Fl5LPl/cZ+O+Pxz2Jlw5ZX2i+9adMzc0HXDYnzGa1LAiz6U0TC9VTy9q7/j3ZBgAAAACJaLYBAAAAQCKabQAAAACQiGYbAAAAACSi2QYAAAAAiWi2AQAAAEAideVy+07yvmTJYeEPXjtuUKHJix4ZX3fA3mG26ktx/3DEux9qX2FvwOZj4jp73RT/fnm/e563n/Tx3HzTF18Ms+2OXllozmpr3G3XMGt96ukqVpKvlo78zrIsO7L+hPZtFlBB1T7WPU8t7QFdZf0Xva+ohIaBA8LsugduCbMTlr8jzDYf9lTunJX4/f/78fvC7Mu7H1hozFdvGRVm2+fcq3xi2fIwu3jM6EK1FFVL6z/Lus4eANVSS3tAV1n/K389PsxGvW9R8vmWnX9QmDV/Lp7v6IXx99mb996pcD1PXrdXmA2Z+Wjhcfnf2rv+PdkGAAAAAIlotgEAAABAIpptAAAAAJCIZhsAAAAAJKLZBgAAAACJaLYBAAAAQCKN7f3Ba8cNSj550WPol79vhzhs3ViwmtgTX5sWZsO/MTvMbmyZF2ZFf/c/tfwkNy86biVqLar1qfg4ZOgIs1oWhNn0polh1jB69zBrW/74NtVUq9pefKmjS+CfvHrLqDDb/uiVyedrHLxb8jHzDJ27fZitnfJqfOGA/mGU/7n6VJgs+/mEnOuybEZTnLV8Mb6XafpOfC/z5d0PzJ2ziKLvi4vHjE5cCWxd6Y5hYVb/ljVVrAQ6n+c/PDXMdr58TvL5vrJyUZh9K74dqYgxZ94bZuWc627ee6fktWRZlg2Z+Wih69adHd8fDP5+fH/A1nmyDQAAAAAS0WwDAAAAgEQ02wAAAAAgEc02AAAAAEhEsw0AAAAAEtFsAwAAAIBE6srlvINp/z9H1p8Q/mDDX4eE17W9+ckwaxi9e3zd8sfbVVc13NgyL8xmNE3uNGNui9eOOyjMtrs2Pta4s/0eRXxgSXxs+1V7xse931a6pq4S9XRWeXsAb9yslgVhNr1pYhUr6T6OevjlMLt1nx3CbMX3poTZHp+dG2a1tAdsXDcyXP9dZa8vqjt8zm2LU5bG92PHbvdcmHWmv03pjvizvP4t8T1Anlpa/1nmHiC1m1oWhtkxTROqWAmNQ5vCrHVtS5jV0h5Qy+v/rOWLw+y80eMKjdmVvgN0pVqLmPbA62E2e/+eYdbe9e/JNgAAAABIRLMNAAAAABLRbAMAAACARDTbAAAAACARzTYAAAAASESzDQAAAAASqSuX23eSb7WP/L2xZV6Ydabj5CvhotV3h9lpIw6pyJyX5Mx5asE5G0cMC7PW1WsKjVkJ9b17h1lp06Ywq6Ujv7Os+nvAmq9OC7Nh58yuYiW1oZb33KJqaQ+o9vrv7kfNdzZ5f+9j3/q+MGt7ZEnyWrrKXlRL6z/Lqr8HUF0XrL4nzM4YcXChMRt2HRRmbU8/U2jMzqSW9gB9gLSe+/jUMBv4kzlVrOTvzlq+OMx+8METw6zunkVhdt3a+8Js5tADw6zar33ROtu7/j3ZBgAAAACJaLYBAAAAQCKabQAAAACQiGYbAAAAACSi2QYAAAAAiWi2AQAAAEAimm0AAAAAkEhjRxfQFTx95rQw2/X82cnnO23EIcnH3JqhjX2Sj9m6ek3yMT+2dGWYXdo8qtCYpU2bipbDNqrff1yYDTsn/doidsDFnwqzYVnXfy0a9moOs7ZHl1axEv4f6z8yNcymN1WxkK24sWVemM1omlxozCeu2TfMhp/wUKExt0UpK4VZ2yNLqlhJ8b9pnmVXTgyzMR9akHw+6GrOGHFw8jHbnn4m+ZhFfWnFg2H27T32q2IltMc+vzgzzEZlc6pYSWXc+tXvxeFX4+ikYenXaZZl2R0v7xVmdfcsKjTmzKEHFrquEvcAeYrW2V6ebAMAAACARDTbAAAAACARzTYAAAAASESzDQAAAAAS0WwDAAAAgEQ02wAAAAAgkcaOnHzVOVPDbEZTfF19375hVtqwYVtK+pd2PX928jFvbJkXZtU+8jbLsmx608RC1529/JEw+/7ovYuWE7q0eVSh61Z+J36vjfpi1z9CuqMVfT+XHlgcZg17NYdZ26NL21dYJ3byktVhdsWeI6pYyd8NOyf9Ppenfv9xYZb3vsiz9PJJYfbYUReHWUfsuWTZ8weUwmxAznWzWhaEWdHPsjx574+ie9+oz78cZq3tKyupomug9fbhYdbwnzuH2c+u/nGYfXT4IYVqyXst6rOc90yW/j1Ta5ZeFu+9zafMr2IlXUfj0PiLTuvalipWUhsO7R3vrN+uYh38f/Lvg4uNucvsncLs2WkvFhu0Av7w6pgwu3bcoCpW8ncPTigXuu6Ja/YNs56z+4XZbucV/M4xZb9i1819sNh1CXiyDQAAAAAS0WwDAAAAgEQ02wAAAAAgEc02AAAAAEhEsw0AAAAAEtFsAwAAAIBE6srl9h31emT9CeEP5h23XvQ4+aLyatn7N2fEF+62OYz2OOn+bSmpW+tMr31Rxy1+Jszyjl++rXRNXSXq6azy9oA8s1oWhNn0pomFajl31dww+9zIKWGW937N01Xey51N/f7jwqz0wOIqVpKv6D5WS3tA3vrPW+MXvrhHmN28906FailP3T/MfvbbC8Pso8MPCbNK7FN5utJeVN+vX5iVXnkl+Xyrf7dvmI1490PJ5yuqltZ/lhW/B1h62aQwaz5lfqFa6nr1CrPy5vh+fsaj68Ns714tYfbtPfZrX2F0Sb9Yc0+Y/fuwg8OslvaAvPX/lZWLwuu+NWp8BaqJlQ47IMxO+smfwuzqsUMqUU5o5XenhtmoL8ypYiU89/H4tRj4k/i1aO/692QbAAAAACSi2QYAAAAAiWi2AQAAAEAimm0AAAAAkIhmGwAAAAAkotkGAAAAAInUlcvtO8m76JHfF62+O8xOG3FImJXuGBZm9W9ZU6SUqruxZV6YzWiaXMVKti7vCOJyQ/zS7/HZuWFW37dvmJU2bAiz5/7YHGYD37E0zIrKe53es3x6fN2hP66ZI7+zrPgekGfp5ZPCrPnD81NPRxcxq2VBmE1vmljFSvK199jv7qAS6z9v7x13/elhNuaT96YupeblrbmjP/CxMGu8I76uqyh6r1ZL6z/LKrMHwL9yU8vCMDumaUIVK8lXS3tAtdd/w17xd8G2R9N/FyRWOuyAMKu/8/7k81W7f3Ld2vvCbObQA8Osvevfk20AAAAAkIhmGwAAAAAkotkGAAAAAIlotgEAAABAIpptAAAAAJCIZhsAAAAAJNJY6QlOG3FIoetmjf1DmM3I0h/7WgmVOJ721icX5eZHDRkfZvlH6RYsKEdpw4ZC1w18R/ojncfnnEyc/zo9HUelwuXwf3v86J+F2eaWLWFWibXVVeSt4yzL/9u8dNPoMNvxmOWFa0ptetPEji6BKsh7ry5puSjMpn+ydt8f27L+l142Kcym59wDNGYLtlpXZ5d//1O7nycdrXTHsDC7ZewNYXZM04RKlNNpHP7Qa2H2y+X579chMx9NXU7VdffXt5YMuKd/mK09d0yY9bn+vkqU0+U1jhqZm7euXBVmRT8H6+/M+RJdAZX4TP7d2jlhNnPo1OTz/f95sg0AAAAAEtFsAwAAAIBENNsAAAAAIBHNNgAAAABIRLMNAAAAABLRbAMAAACAROrK5XK7fvDI+hPCH9zwroPC6/r+4d43XtVW1DU2hlm5tbXQmCu/Gx/7OuoL8XGxXcmqb8W/4z0f+F6YnTTs4EqUE5rVsiDMpjdNDLO890VeVtq0qX2F/ZPbStfUFbqwi9q8blS4B5SyUnhdJY5wroSiR2J3F53p9y9ay4mPPRlmV48dsk01/Su1tAeM+8p54frv/Xx8HzHwJ13j87Mzvf+3xYrvTQmzgffH1+385xVhdt2im8Ks6N9m4627h1mfox4vNObTZ0wLs10vmF1ozDy1tP6zLMsOede54ULvu+a18LrygkcqUk9qSy+bFGbNp8yvYiUdo2Gv5jBre3RpFSvJsiufuDvMPjT8kDBbennOa/jh9K9hLe0BeX2AbMp+8YVzH6xANcV8beXCMPvGqAlVrKRyTnpsbZj9auzQMMu7Bxp74yfDrPkT97WvsDegaB+goXmPMGtbGt/jFNXe9e/JNgAAAABIRLMNAAAAABLRbAMAAACARDTbAAAAACARzTYAAAAASESzDQAAAAASqSuX45N8//9yj/ztIhr2HB1mpb69wqx8f3xseX3v3vGYmzaF2csnTgmzHa6eG2bdRcO4MWHWtnhZ8vle+ODUMOv/8zmFxqylI7+zrHvsAXlHW89omlzFSop7+X3x3pFlWfa3cy8Is870OxZ9LTrTa1hLe0DR9V/ft2+YlTZsKFxPEevOnhZmQ3++JMzanlsfZkXfjy+dFK/jHX8V3wPMalkQZlmWZXvdeUqYjXrfojBb+pO41uaPx79jLaul9Z9l3eMe4KaWhWF2TNOEKlZS3NLLJ+XmA+7pEWeXFbvfrYSir8War8b7+LBzZm9TTW9ULe0B3WH9V8KJjz0ZZlePHVLFSrqWV084KMy2v+be5PP9as09YXbSsIMLjdne9e/JNgAAAABIRLMNAAAAABLRbAMAAACARDTbAAAAACARzTYAAAAASESzDQAAAAASqSuX23eSb96Rv5umHxhe13vWfW+8qizLnjktPtp50EXVPdo5z40t88JsRtPkQmP+cFX8+43qER/pvS1zFnXc4mfC7Npxg6pYSf5r8baTPxFmPf48v9B8tXTkd5ZV5tjvznSEe2fSsGu8dtqejtdcR3jlvVPCrN9v5laxkiyb1bIgzKY3TUw+Xy3tAXnrvxKfg11F3u/+9pM+HmYNf1tYaMzO9vdcdsFBYTbmjHurWEl+LecedXWYXdo8KszyXos+g1fVzPrPssrcA9D1lQ47IMzq77y/ipVk2U0t8b56TNOE5PO5B/i7W59cFF531JDxyWt59tSpYbbLJXOSz0fXsN3/2SXMrt7jpjArel/V3vXvyTYAAAAASESzDQAAAAAS0WwDAAAAgEQ02wAAAAAgEc02AAAAAEhEsw0AAAAAEqkrl9t3knfekb+fWLY8vO7SY48Js7ZHl7Zr7o72lZWLwuxbo8aHWV1jY5iVW1vD7MUPxEca73RV/pHGs1oWhNn0pom51/K/NQwcEGa3PHNJzRz5nWX5ewBv3NIr4vXYfHK8jrfmxpZ5YVb0eGv+tfYe+90d5K3/vPfcuD+cHmZjzrh324qqkk3vODDMev/xvjBraN4jzNqWrgiz5z4W3wMMvLQy9wAtX5wWZk3fmR1m3WG/yfsd8vQZvKpm1n+WuQfoKm5qWRhmxzRNqGIl3Z97ADqb+r59w6y0YUPy+d69+Kkw+9243ZLPVwlrr907zEZ8PP792tsH8GQbAAAAACSi2QYAAAAAiWi2AQAAAEAimm0AAAAAkIhmGwAAAAAkotkGAAAAAInUlcvtO8m36JG/1T4WvjvMV3dAfARt+f5HCo251Tkn7xtmP7v24jDbpaFXmL3r0BPCrHXlqnbV9Uas/M7UMBv1xTmFxqz7S1OY/fnfflgzR35nWf4eMKtlQXjd9KaJFamH6vH6/mu3la6pmT0gb/3nfQ7mqcRncmfSuPuIMGt9fHWYDbinf5itP/iFbaopUvQ13OdXZ4bZoyf9OMwqsW9U+/6vltZ/luXvAfXbbRdeV3rttYrUQ/Xc1LIwzI5pmlDFSjqXWtoDivYBnvtjc5gNfMfSwvVEGvYcHWZtS5Ynn6/WbXjXQWHW9w/3VrGS6mvv+vdkGwAAAAAkotkGAAAAAIlotgEAAABAIpptAAAAAJCIZhsAAAAAJKLZBgAAAACJ1JXL7TvJt+iRv7mT/6UpzMqHt6SeLqvr1SvMblh5d5gVPTJ+4v2lMFtwQNfpcy696MAwq98c/x6jz5pbiXI6jVo68jvLsmzzulHhHjC9aWI1S6mIG1vmhVnRPaDaY27LuJ1JJf5ulVBLe8BR/T4Urv/Sa69Vs5Rc5an7x2F9/HLV3bMofTEVcMnq+F4ly7Ls1BGHVKmSyrn1yUVhdtSQ8VWrY2tqaf1nWWW+B3QmN7UsDLNjmiZ0mjGX//KA3Hz0++8vNG5nMmZe/H1t2eTNVawkXy3tAd1h/W/t/jnSmb4DdISTl6wOsyv2HFHFSjqX9q7/rtPxAQAAAIBOTrMNAAAAABLRbAMAAACARDTbAAAAACARzTYAAAAASESzDQAAAAASqSuX23eSb9Ejf7vKsbdF6+wqv9+2aBy8W5i1rnsqzCrxt2nYYYcwa3v55UJj5sn7HfoMXlUzR35nWZZtXjcq3AOmN02sZinwhs1qWRBmM455f5iVHlgcZu099rs72P/M88L1P+jHs8Pr6nr1CrPy5s3bVtQbNHTu9mG27sNDwqzt0aWVKKfbq8Q9QNExf7gqfo9+euS0QvPV2j3AlnV7hHvAMU0TqlkKvGEXrb47zE4bcUihMWvpHqBoH6AzWXrZpDD7+Zt/FmbfGjU+zGqhD1BUJf42tz65KMyOGjK+0Jh5zlz+WJi9Y9RD7Vr/nmwDAAAAgEQ02wAAAAAgEc02AAAAAEhEsw0AAAAAEtFsAwAAAIBENNsAAAAAIJHGFINcknOc8r2bt08xRbsVPWY29wjaO4bmXNeusrq01nVPFbquEkceH3L302F25359wuzs5Y+E2Zv7bAqz6Tm/w22lMOqWpjdN7OgS6CCd6WjzWS0Lwqzoe7T0wOKi5dSMQT+eHWYvvX9KmN313QvDrBJ7yrILD4rDKfeGUX3vJ8KsM73/u5JK/G3eccJHwqwueyDMRvfoFWZ5r++56/cNs68NDqNu6ZimCR1dAh2kbnK8DsrzHqpiJVl2wep7wuyMEQeH2anLTwyz+mzNNtVE19B8yvww++Bl8WfLuxYsDDP3ALG8v82v1sTr+KRh8TreUm4rVEve53zemMcNje9v39HOPoAn2wAAAAAgEc02AAAAAEhEsw0AAAAAEtFsAwAAAIBENNsAAAAAIBHNNgAAAABIpK5cLnd0DQAAAADQLXiyDQAAAAAS0WwDAAAAgEQ02wAAAAAgEc02AAAAAEhEsw0AAAAAEtFsAwAAAIBENNsAAAAAIBHNNgAAAABIRLMNAAAAABL5vwBxRrvGorNs9gAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 1600x1200 with 20 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "q14jDKXq--_e"
      },
      "source": [
        "# ConvNet with permuted pixels"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9n46s_sr--_e",
        "outputId": "659f40e3-3278-43f8-99e5-435bd8df706c"
      },
      "source": [
        "# Training settings \n",
        "n_features = 6 # number of feature maps\n",
        "\n",
        "model_cnn = CNN(input_size, n_features, output_size)\n",
        "model_cnn.to(device)\n",
        "optimizer = optim.SGD(model_cnn.parameters(), lr=0.01, momentum=0.5)\n",
        "print('Number of parameters: {}'.format(get_n_params(model_cnn)))\n",
        "\n",
        "for epoch in range(0, 1):\n",
        "    train(epoch, model_cnn, perm)\n",
        "    test(model_cnn, perm)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Number of parameters: 6422\n",
            "Train Epoch: 0 [0/60000 (0%)]\tLoss: 2.308466\n",
            "Train Epoch: 0 [6400/60000 (11%)]\tLoss: 2.285470\n",
            "Train Epoch: 0 [12800/60000 (21%)]\tLoss: 2.220113\n",
            "Train Epoch: 0 [19200/60000 (32%)]\tLoss: 1.891632\n",
            "Train Epoch: 0 [25600/60000 (43%)]\tLoss: 1.541811\n",
            "Train Epoch: 0 [32000/60000 (53%)]\tLoss: 1.097137\n",
            "Train Epoch: 0 [38400/60000 (64%)]\tLoss: 0.940273\n",
            "Train Epoch: 0 [44800/60000 (75%)]\tLoss: 0.822390\n",
            "Train Epoch: 0 [51200/60000 (85%)]\tLoss: 0.632361\n",
            "Train Epoch: 0 [57600/60000 (96%)]\tLoss: 0.462242\n",
            "\n",
            "Test set: Average loss: 0.5808, Accuracy: 8113/10000 (81%)\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8R0Qecjs--_e"
      },
      "source": [
        "# Fully-Connected with Permuted Pixels"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "UOPc_l7p--_e",
        "outputId": "08546902-7099-4d85-94a9-64fcb48bd8b3"
      },
      "source": [
        "n_hidden = 8    # number of hidden units\n",
        "\n",
        "model_fnn = FC2Layer(input_size, n_hidden, output_size)\n",
        "model_fnn.to(device)\n",
        "optimizer = optim.SGD(model_fnn.parameters(), lr=0.01, momentum=0.5)\n",
        "print('Number of parameters: {}'.format(get_n_params(model_fnn)))\n",
        "\n",
        "for epoch in range(0, 1):\n",
        "    train(epoch, model_fnn, perm)\n",
        "    test(model_fnn, perm)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Number of parameters: 6442\n",
            "Train Epoch: 0 [0/60000 (0%)]\tLoss: 2.325015\n",
            "Train Epoch: 0 [6400/60000 (11%)]\tLoss: 1.674272\n",
            "Train Epoch: 0 [12800/60000 (21%)]\tLoss: 0.956756\n",
            "Train Epoch: 0 [19200/60000 (32%)]\tLoss: 0.812652\n",
            "Train Epoch: 0 [25600/60000 (43%)]\tLoss: 0.835974\n",
            "Train Epoch: 0 [32000/60000 (53%)]\tLoss: 0.470191\n",
            "Train Epoch: 0 [38400/60000 (64%)]\tLoss: 0.562551\n",
            "Train Epoch: 0 [44800/60000 (75%)]\tLoss: 0.345490\n",
            "Train Epoch: 0 [51200/60000 (85%)]\tLoss: 0.521773\n",
            "Train Epoch: 0 [57600/60000 (96%)]\tLoss: 0.468404\n",
            "\n",
            "Test set: Average loss: 0.3879, Accuracy: 8846/10000 (88%)\n",
            "\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "rCktvCVG--_f"
      },
      "source": [
        "# The ConvNet's performance drops when we permute the pixels, but the Fully-Connected Network's performance stays the same\n",
        "\n",
        "* ConvNet makes the assumption that pixels lie on a grid and are stationary/local\n",
        "* It loses performance when this assumption is wrong\n",
        "* The fully-connected network does not make this assumption\n",
        "* It does less well when it is true, since it doesn't take advantage of this prior knowledge\n",
        "* But it doesn't suffer when the assumption is wrong"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "CFtKH52e--_f",
        "outputId": "8ef5f007-5b52-41fa-f68c-73dba665c12d"
      },
      "source": [
        "plt.figure(figsize=[16,6])\n",
        "plt.bar(('NN image', 'CNN image',\n",
        "         'CNN scrambled', 'NN scrambled'),\n",
        "        accuracy_list, width=0.4)\n",
        "plt.ylim((min(accuracy_list)-5, 96))\n",
        "plt.ylabel('Accuracy [%]')\n",
        "for tick in plt.gca().xaxis.get_major_ticks():\n",
        "    tick.label.set_fontsize(20)\n",
        "plt.title('Performance comparison');"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAABSgAAAITCAYAAAAEi/JoAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAA9hAAAPYQGoP6dpAABRk0lEQVR4nO3dd7gtVX0//vcHLmKBi11EReyJsQCCBqOCii0Y6zdijdhiJfaCWFAx2IP+1ERjQWxIsPcWsYNdsaBEilJVVECEi8D6/TFzPJvD3vvUy9x7z+v1PPPsMzNr1qyZfc46sz97lWqtBQAAAABgCJsNXQAAAAAAYPUSoAQAAAAABiNACQAAAAAMRoASAAAAABiMACUAAAAAMBgBSgAAAABgMAKUAAAAAMBgBCgBAAAAgMEIUAIAAAAAgxGgBACYoqr2qao2slxYVSdX1Tur6jorfK7LVdV/VdVpVXVRVf1gJfNn9aiqPfrf1z2GLgsAwHzWDF0AAICNxKOSHJvkCknulGS/JLtX1S1ba+eu0DmemOTxSfZN8t0kf1qhfFl9vpdktyQ/HbogAADzEaAEAFiYH7fWvtP//KWq2jzJC5PcL8l7l5NxVV2xtfbnJLdIcl5r7Y3LKukl875Ca+28lcqPDVtVbZGktdbOTnLU0OUBAFgIXbwBAJZmJvhz/SSpzpOq6gdVdV5V/aGqjqiqG44eVFVHVtWPq+pOVfWNqvpzkndUVUvy2CRXGOlOvk9/zOWr6qCqOqGqLqiqU6rqTVV15Tl5n1hVn6iqB1TV96vq/CQvHunu+9CqemXfhfxPVfXxqrpWVW1dVW+tqt/1yzuraqs5eT+5qr5SVb+pqnOr6piqek4fEBt3fbtW1Ver6s9VdXxVPa+qNpuT9spV9dp+/7o+709V1d+MpLlcVb2gqo7t0/y2L981FvImVdXt+us8s6rOr6pfVtXBc9Lcoaq+WFXn9OX9RlXtNSfNTFf/u1TVf/f5nV1Vh1bVlapq26o6vKr+2N/f14zem6raoT/+OVW1f1X9qi/Pd6rqrnPOdeP+Go/ry3NKfw23nJNu5n19RH8fT0myLsmNx3XxrqobVtVhVXVqfy/P6K97x5E0m/VlnLnfv+mv8bpLfZ8BAOajBSUAwNLcuH/9bf/6liT7JHlDkucmuWqSFyX5RlXdurV2xsix107yniSvSvL8JBcnOThdi8w7J7lLn+6XVVVJPpLkrkkOSvLVJLdK8pIku1XVbq21dSN575zkb5McmOSEJOcmuVK/79+TfKkv5w5JXpPk/UkuTPLDJA9JslOf7pwk/zaS742SvK/P84Ikt06yf5K/SfLoOfdm23StSl/bl/P+fdlPTXJoklTV1km+1pfjlUmOTrJVuu7z105ybB/o+miSO/b36hvpAsIvSXJkVe0yrXVoVd0jyceT/CzJM5L8qj/f3UfS7J7k80l+lOQx6QJ8T0ry8ap6SGvtA3OyfVuSDyV58Mi9WpPkZv32tybZM93vwKlJXjfn+KckOSnJ09I1FnhOkk9X1e6ttW/2abZLcmaS56X7/bpqkkcmObqqdmqt/XxOngcl+WaSJ6T7XfpNuvdgrk8l2bw/56+SXD3J7ZNceSTNfyb51yRvTPKJ/n69LMkeVbVza+13I2nnfZ8BABaktWaxWCwWi8VimbCkC+a1JLdLF4jaKsle6YJAZye5VpK/79M8Y86x103y5ySvHNl2ZJ/2LmPOdUiSP83Zdo8+/bPnbH9Qv/1xI9tOTBdsvOmctHv0aT82Z/t/9NtfP2f7h5OcOeWebNbfi0f057vKmOu77ZxjfpLkMyPrL+zT7TnlPA/u0zxgzvZd+u1PnOe9+79+ufyUNN9MckaSrUa2bZ7kmCS/TlJzfg/eMOZetSRPn7P9+0m+O7K+Q5/ulNHyJNk6XTDy81PKuHmSLZL8IsnrxryvXx5zzMy+Pfr1q/XrT51ynr/p07xpzvbb9ttfvtj32WKxWCwWi2Uhi+4XAAALc1SSv6RrWfiJJKcnuVfrWkbeO12w5j1VtWZm6dP8MF2waNQfWmv/u8DzzrSmPGTO9v9J1zryrnO2/6i19osJeX1izvrP+tdPjtl+1dFu3lW1U1V9rKrOTHJRuntxaLrg2U3nHH96a+1bc8uVvjt8715JftFa+8KEsibdff1jutaMo/f1B+nu7R6TDqyqm6Zr9fn21tr5E9JcKV3g+YjW2l8nJGqtXZTk3ekCzDebc9hi7uH1c2kfGi1Pa+2cdK0871TduKbpr/P5VfXTqrogXRD4giQ3Sdc6dq4Pjru+OX6f5JdJnl1Vz+jfz7mfBe7cvx4yurF/L3+WS/+uLeR9BgCYly7eAAAL8y/pgjQXJjmjtXbayL5rJal0LfHGOX7O+mljU413tSQXttZ+O7qxtdaq6vR+/0Lz/v2c9Qvm2X75JH+qqu3TdS3/eZKnpmupeX66lnVvSjez+agzx5x73Zx010jXzXiaa6XrfnzBhP1Xn3LszBiVJ09Jc5V079u4e3Zq/zr3/i7mHl5+TL6nT9h2uXStc89K1y38yem6vn85yR/Sdd1+Wy59r5MF/D71vy93TTfswHPSdcv+fVW9N8n+faB05lon3Y+5gceFvM8AAPMSoAQAWJiftdlZvOf6XboWlHdMF6CZa+62tojznplkTVVdYzRI2Y9NuW2Sby8j74W6X7pxLB/QWjtppAw7LiPP36ZroTjN79Jd/z0n7D9nnvwzzzlmAn/XHrNvu5EyrKRxY0Numy6gOdOK8+FJDm2tPX80UVVdPV2L0rkW9J73791j+rxumm6YgAPSBUefkNmA47Vz6cDudln5ewEAkMQs3gAAK+ET6VriXae19p0xyzHLyPuL/evD52x/YLqg4Rez/s0EwP4aaO0DpI9bRp6fTnLTqrrLlDSfSNeqb/MJ93XuZDGzBe66uf8yyaOrassJac5NNznPA6rqr63++q7PD08XpJvUXX6pHlBVf21Z2U8W9E9Jvtp3LU+6+32JoHY/q/h1VqoQrbVftNYOTDfW5s795plhBy7xu1ZVu6brWn5Z/K4BAKuQFpQAAMvUWvt6Vb01yTurapckX0k3PuS1k9whyTGttf9cYvafT/LZJK+sqrVJvp7ZWby/n26sxPXt8+la+L2/ql6VruvyE9N1kV6qg5PsneSjVfWKJN9K1zV49ySfaK19KclhSR6W5FNV9fo+zV/StYq8c5KPttY+POUcT043vuNRVfUf6bqUb5/kHq21h/Vp9uuv70tV9Zr+Op+U5BZJHtJaW+kWqRcl+XxVvS5dY4HnJlmb5MUjaT6RZJ+qOjbdmI63SfLsTO+uPlVV3SrdzNz/k+S4dNd5l3S/S69Iktbaz/vf432r6uJ0QeQd0s3i/et0kyoBAKw4AUoAgBXQWnt8VR2V5PHpAlybpRu37+vpAmtLzbdV1f3SdcV9VJL903W1fXeS57fWxnUpX1GttWOr6oFJDkzyoXRdgd+XbqzETy8xz3Oq6g7prutf0wXo/pCuy/pb+zQXVdV90o17+Yh0wcQL0wXqvpyu9d+0c3y2qu6UbtzFN6QLrJ6c5GMjab7ct+J8SbrJYTZLN7HRfVprcyfEWQlv7MvxhiTXTDfr9V6tta+PpHlqukDsfunGpfxekgeku/9LdXq6FqVPSnK9dK00j0/yzCT/30i6J/bpHpMuwHtWks8k2a+1Nm7MSQCAZauV/1IYAAAYVVU7JDkhybNba68ZuDgAABsUY1ACAAAAAIMRoAQAAAAABqOLNwAAAAAwGC0oAQAAAIDBCFACAAAAAIMRoAQAAAAABrNm6AJsiKqqkmyX5JyhywIAAAAAG6mtk5za5pkER4ByvO2SnDx0IQAAAABgI3fdJKdMSyBAOd45SfLrX/86a9euHbosAAAAALBROfvss3O9610vWUAPZQHKKdauXStACQAAAADrkUlyAAAAAIDBCFACAAAAAIMRoAQAAAAABiNACQAAAAAMRoASAAAAABiMACUAAAAAMBgBSgAAAABgMAKUAAAAAMBgBCgBAAAAgMEIUAIAAAAAgxGgBAAAAAAGI0AJAAAAAAxGgBIAAAAAGIwAJQAAAAAwGAFKAAAAAGAwApQAAAAAwGAGDVBW1dZVdXBVnVRV51XVN6pq15H9h1RVm7MctYB8H1hVP62qdf3r/dfvlQAAAAAASzF0C8q3JblbkkckuWWSzyX5QlVdZyTNZ5Jce2T5x2kZVtVuST6Q5N1Jbt2/Hl5Vt1vx0gMAAAAAy1KttWFOXHWFJOckuW9r7ZMj23+Q5BOttRdU1SFJrtxau98i8v1AkrWttXuNbPtMkj+01h6ywDzWJjnrrLPOytq1axd6agAAAAAgydlnn51tttkmSbZprZ09Le2QLSjXJNk8yflztp+X5A4j63tU1W+q6hdV9d9Vdc158t0tXUvMUZ9NcvtJB1TVllW1dmZJsvXCLgEAAAAAWI7BApSttXOSfDPJC6tqu6ravKoenuR26bpyJ8mnkzwsyV2SPDPJrkn+t6q2nJL1tknOmLPtjH77JPslOWtkOXmRlwMAAAAALMHQY1A+IkklOSXJuiT/luR9SS5KktbaB1prn2yt/bi19vEk90py0yR7zZPv3H7rNWbbqIOSbDOyXHeR1wEAAAAALMGaIU/eWvtlkt2r6krpxo08rR9D8oQJ6U+rqpOS3GRKtqfn0q0lr5lLt6oczXddugBpkqSqFngFAAAAAMByDN2CMknSWju3Dz5eJck9knx0XLqqulqS6yU5bUp230w3M/iouyf5xkqUFQAAAABYOYO2oKyqe6Trfv3zJDdO8ur+53dW1VZJDkjywXQByR2S/HuS3yX58EgehyY5pbW2X7/p9Um+UlXPTRfovG+SPXPJiXcAAAAAgA3A0C0ot0nypiTHJjk0ydeS3L219pd041DeMl2Q8RdJ3tW/7tZPsDNj+8xOqpPW2jeSPDjJo5L8KMk+SfZurR29vi8GAAAAAFicam3a3DGrU1WtTXLWWWedlbVr1w5dHAAAAADYqJx99tnZZpttkmSb1trZ09IO3YISAAAAAFjFBCgBAAAAgMEIUAIAAAAAgxGgBAAAAAAGI0AJAAAAAAxGgBIAAAAAGIwAJQAAAAAwGAFKAAAAAGAwApQAAAAAwGAEKAEAAACAwQhQAgAAAACDEaAEAAAAAAYjQAkAAAAADEaAEgAAAAAYjAAlAAAAADAYAUoAAAAAYDAClAAAAADAYAQoAQAAAIDBrBm6AACwIdvheZ8cughsYE58xV5DFwEAADYpWlACAAAAAIMRoAQAAAAABiNACQAAAAAMRoASAAAAABiMACUAAAAAMBgBSgAAAABgMAKUAAAAAMBgBCgBAAAAgMEIUAIAAAAAgxGgBAAAAAAGI0AJAAAAAAxGgBIAAAAAGIwAJQAAAAAwGAFKAAAAAGAwApQAAAAAwGAEKAEAAACAwQhQAgAAAACDEaAEAAAAAAYjQAkAAAAADEaAEgAAAAAYjAAlAAAAADAYAUoAAAAAYDAClAAAAADAYAQoAQAAAIDBCFACAAAAAIMRoAQAAAAABiNACQAAAAAMZtAAZVVtXVUHV9VJVXVeVX2jqnbt921RVa+sqmOq6tyqOrWqDq2q7ebJc5+qamOWy182VwUAAAAALNSagc//tiS3SPKIJKcmeXiSL1TVzZP8KcnOSV6W5IdJrpLk4CQfS7LLPPmeneRmoxtaa+evZMEBAAAAgOUbLEBZVVdI8sAk922tfaXffEBV3S/JE1trL0hytznH7JvkW1W1fWvtV1Oyb62109dHuQEAAACAlTNkF+81STZPMrdl43lJ7jDhmG2StCR/nCfvrfpu4ydX1Seqaqdpiatqy6paO7Mk2Xr+4gMAAAAAyzVYgLK1dk6SbyZ5YVVtV1WbV9XDk9wuybXnpu/HkHxFkve11s6ekvWxSfZJcp8kD0kXAP16Vd1kyjH7JTlrZDl58VcEAAAAACzW0LN4PyJJJTklybok/5bkfUkuGk1UVVskOSxdeZ80LcPW2lGttfe01n7YWvtqkgcl+UWSfaccdlC61pkzy3WXdDUAAAAAwKIMOklOa+2XSXavqislWdtaO62qPpDkhJk0fXDy8CQ3SHKXeVpPjjvHxVX17SQTW1C21talC5DOnHNxFwIAAAAALMnQLSiTJK21c/vg5FWS3CPJR5NLBCdvkmTP1tqZi827umjjjklOW7kSAwAAAAArYdAWlFV1j3RdvH+e5MZJXt3//M6qWpPkiCQ7J7l3ks2ratv+0N+31i7o8zg0ySmttf369RcnOSrJcUnWpus2vmOSJ19GlwUAAAAALNCgAcp04z0elG7Mx98n+WCS/Vtrf6mqHdJNdJMkP5hz3J2THNn/vH2Si0f2XTnJW5Nsm27Cm+8nuVNr7VsrXnoAAAAAYFmGHoPy8HRduMftOzFd68r58thjzvrTkzx9BYoHAAAAAKxnG8QYlAAAAADA6iRACQAAAAAMRoASAAAAABiMACUAAAAAMBgBSgAAAABgMAKUAAAAAMBgBCgBAAAAgMEIUAIAAAAAgxGgBAAAAAAGI0AJAAAAAAxGgBIAAAAAGIwAJQAAAAAwGAFKAAAAAGAwApQAAAAAwGAEKAEAAACAwQhQAgAAAACDEaAEAAAAAAYjQAkAAAAADEaAEgAAAAAYjAAlAAAAADAYAUoAAAAAYDAClAAAAADAYAQoAQAAAIDBCFACAAAAAIMRoAQAAAAABiNACQAAAAAMRoASAAAAABiMACUAAAAAMBgBSgAAAABgMAKUAAAAAMBgBCgBAAAAgMEIUAIAAAAAgxGgBAAAAAAGI0AJAAAAAAxGgBIAAAAAGIwAJQAAAAAwGAFKAAAAAGAwApQAAAAAwGAEKAEAAACAwQhQAgAAAACDEaAEAAAAAAYjQAkAAAAADEaAEgAAAAAYjAAlAAAAADAYAUoAAAAAYDAClAAAAADAYAYNUFbV1lV1cFWdVFXnVdU3qmrXkf1VVQdU1an9/iOr6u8WkO8Dq+qnVbWuf73/+r0SAAAAAGAphm5B+bYkd0vyiCS3TPK5JF+oquv0+5+T5BlJnpJk1ySnJ/l8VW09KcOq2i3JB5K8O8mt+9fDq+p26+siAAAAAIClGSxAWVVXSPLAJM9prX2ltfZ/rbUDkpyQ5IlVVUmeluTlrbUPtdZ+nOSRSa6Y5KFTsn5aks+31g5qrR3bWjsoyRf77QAAAADABmTIFpRrkmye5Pw5289LcockN0iybbpWlUmS1tq6JF9Ocvsp+e42ekzvs9OOqaotq2rtzJJkYgtNAAAAAGDlDBagbK2dk+SbSV5YVdtV1eZV9fAkt0ty7XTBySQ5Y86hZ4zsG2fbJRyzX5KzRpaTF3QRAAAAAMCyDD0G5SOSVJJTkqxL8m9J3pfkopE0bc4xNWbbXIs95qAk24ws150nfwAAAABgBQwaoGyt/bK1tnuSrZJcr7V22yRbpBuH8vQ+2dyWj9fMpVtIjjp9sce01ta11s6eWZKcs4jLAAAAAACWaOgWlEmS1tq5rbXTquoqSe6R5KOZDVLebSZdVV0uye5JvjElu2+OHtO7+zzHAAAAAAADWDPkyavqHum6X/88yY2TvLr/+Z2ttVZVByd5flUdl+S4JM9P8ud03cBn8jg0ySmttf36Ta9P8pWqem66QOd9k+yZbuIdAAAAAGADMmiAMt14jwelG/Px90k+mGT/1tpf+v2vSnKFJG9OcpUkRye5ez/Bzoztk1w8s9Ja+0ZVPTjJgUleluSXSfZurR29nq8FAAAAAFikQQOUrbXDkxw+ZX9LckC/TEqzx5htRyQ5YtkFBAAAAADWqw1iDEoAAAAAYHUSoAQAAAAABiNACQAAAAAMRoASAAAAABiMACUAAAAAMBgBSgAAAABgMAKUAAAAAMBgBCgBAAAAgMEIUAIAAAAAgxGgBAAAAAAGI0AJAAAAAAxGgBIAAAAAGIwAJQAAAAAwGAFKAAAAAGAwApQAAAAAwGAEKAEAAACAwQhQAgAAAACDEaAEAAAAAAYjQAkAAAAADEaAEgAAAAAYjAAlAAAAADAYAUoAAAAAYDAClAAAAADAYAQoAQAAAIDBCFACAAAAAIMRoAQAAAAABiNACQAAAAAMRoASAAAAABjMmqELAAAAALAa7PC8Tw5dBDYwJ75ir6GLsEHQghIAAAAAGIwAJQAAAAAwGAFKAAAAAGAwApQAAAAAwGAEKAEAAACAwQhQAgAAAACDEaAEAAAAAAYjQAkAAAAADEaAEgAAAAAYzJqhC8BwdnjeJ4cuAhuQE1+x19BFAAAAAFahBQUoq+p7i8y3JblPa+2UxRcJAAAAAFgtFtqCcsckr03ypwWkrSTPS7LlEssEAAAAAKwSi+ni/erW2m8WkrCqnrnE8gAAAAAAq8hCA5Q3SPLbReR78ySnLr44AAAAAMBqsqAAZWvtpMVk2lr79dKKAwAAAACsJkuexbuq1iR5fJI9kmye5OtJ3tRaO39ligYAAAAAbOqWHKBM8oYkN03yoSRbJPmXJLskecgKlAsAAAAAWAU2W2jCqrr/nE13T3KP1tqbW2uvT/KwJPdazMmrak1VHVhVJ1TVeVV1fFW9qKo2G0nTJizPnpLvPhOOufxiygcAAAAArF+LaUH5mKp6ZJInt9ZOSfK9JP9VVR9M14LycUm+vcjzPzfJE5I8MslP0rXAfGeSs5K8vk9z7TnH3CvJ25N8cJ68z05ys9ENup8DAAAAwIZlwQHK1tq9q+rBSY6sqjck+dckL0zy8syOQXnAIs+/W5KPttY+2a+fWFUPSReonDnv6aMHVNV9k3yptXb8/EW+5LEAAAAAwIZlwV28k6S1dliSXZPcKslnk7y7tXab1tqOrbUnt9Z+u8jzfy3JXavqpklSVbdOcocknxqXuKqulWSvdC0o57NVVZ1UVSdX1SeqaqdJCatqy6paO7Mk2XqR1wEAAAAALMGiApRJ0lr7Y2vtcUmeneTdVfXqqrrCEs//yiTvT3JsVf0lyfeTHNxae/+E9I9Mck66iXmmOTbJPknuk27SnvOTfL2qbjIh/X7pupXPLCcv4hoAAAAAgCVazCQ516uqD1TVMVX13iTHJblNkvOS/KCqFjVBTm/vJA9P8tAkO6cLQD6rH+tynEcnee98Y0m21o5qrb2ntfbD1tpXkzwoyS+S7DvhkIOSbDOyXHfRVwIAAAAALNpiWlAemqSlazn5myRvaa1d0Fp7UZL7Jdmvqg5f5PlfneQVrbXDWmvHtNbeneQ/0rVovISqumO6SW/etshzpLV2cboJfMa2oGytrWutnT2zpGulCQAAAACsZ4uZxXuXJDu21n5ZVZ9NcsLMjtbaz5Lcqar+dZHnv2KSi+dsuyjjA6ePSfLd1toPF3mOVFUl2THJMYs9FgAAAABYfxYToPxekpdW1buS7Jkxwb7W2lsXef6PJ9m/qn6V5CdJdkryjCTvGE3UT1zzz0meOS6Tqjo0ySmttf369RcnOSpdN/S1Sf4tXYDyyYssHwAAAACwHi0mQPkvSV6brgv2D5I8fgXOv2+SlyV5c5JrJjk1yVuSvHROugcnqXQT6oyzfS7ZEvPKSd6aZNt0k958P8mdWmvfWoEyAwAAAAArZMEBytbaSUn+30qevLV2TpKn9cu0dG9NF3CctH+POetPT/L0ZRcQAAAAAFivFjRJTt/FesGqauulFQcAAAAAWE0WOov3H6rqmovI95SquuFSCgQAAAAArB4L7eJdSR5bVX9aYPotllgeAAAAAGAVWWiA8ldJHreIfE9P8pfFFwcAAAAAWE0WFKBsre2wnssBAAAAAKxCCx2DEgAAAABgxQlQAgAAAACDEaAEAAAAAAYjQAkAAAAADEaAEgAAAAAYzKIDlFV1YlW9qKq2Xx8FAgAAAABWj6W0oHxtkvsmOb6qPl9VD66qLVe4XAAAAADAKrDoAGVr7f9rrd0myW2S/DTJG5KcVlVvrKqdV7qAAAAAAMCma8ljULbWfthae2qS6yR5SZLHJvl2Vf2wqh5dVbVShQQAAAAANk1rlnpgVW2R5P5JHpXkbkmOSvL2JNsleXmSPZM8dAXKCAAAAABsohYdoOy7cT8qyUOSXJTk3Ume3lo7diTN55J8ZaUKCQAAAABsmpbSgvLbST6f5IlJPtJa+8uYND9NcthyCgYAAAAAbPqWEqC8YWvtpGkJWmvnpmtlCQAAAAAw0VImyblmVd1u7saqul1V7bICZQIAAAAAVomlBCjflOR6Y7Zfp98HAAAAALAgSwlQ3jzJ98Zs/36/DwAAAABgQZYSoFyX5Fpjtl87yYXLKw4AAAAAsJosJUD5+SQHVdU2Mxuq6spJ/r3fBwAAAACwIEuZxfuZSb6S5KSq+n6/bcckZyR5xAqVCwAAAABYBRYdoGytnVJVt0rysCS3TnJekncmeX9r7S8rXD4AAAAAYBO2lBaUaa2dm+StK1wWAAAAAGCVWVKAMkmq6uZJtk9yudHtrbWPLbdQAAAAAMDqsOgAZVXdMMmHk9wySUtS/a7Wv26+MkUDAAAAADZ1S5nF+/VJTkhyrSR/TvJ3Se6U5DtJ9lixkgEAAAAAm7yldPHeLcldWmu/raqLk1zcWvtaVe2X5A1JdlrREgIAAAAAm6yltKDcPMmf+p9/l2S7/ueTktxsJQoFAAAAAKwOS2lB+eMkt0pyfJKjkzynqi5I8q/9NgAAAACABVlKgPLAJFfqf35Bkk8k+WqSM5PsvULlAgAAAABWgUUHKFtrnx35+fgkN6+qqyb5Q2utTT4SAAAAAOCSFjUGZVWtqaoLq+oWo9tba78XnAQAAAAAFmtRAcrW2oXpJsPZfP0UBwAAAABYTZYyi/eBSQ7qu3UDAAAAACzZUibJ+bckN05yalWdlOTc0Z2ttZ1XomAAAAAAwKZvKQHKj6x0IQAAAACA1Wkps3i/ZH0UBAAAAABYfZYyBiUAAAAAwIpYdAvKqro4SZu0v7Vmhm8AAAAAYEGWMgbl/eesb5FkpySPTPLiZZcIAAAAAFg1ljIG5UfHbD6iqn6SZO8kb192qQAAAACAVWElx6A8OsmeK5gfAAAAALCJW5EAZVVdIcm+SU5eifwAAAAAgNVh0QHKqvpDVf1+ZPlDknOSPDrJsxeZ15qqOrCqTqiq86rq+Kp6UVVtNpLmkKpqc5ajFpD3A6vqp1W1rn+dO3YmAAAAADCwpUyS8/Rcchbvi5P8NsnRrbU/LDKv5yZ5QroJdn6SZJck70xyVpLXj6T7TJJHjaxfMC3TqtotyQeSvDDJh9NN7HN4Vd2htXb0IssIAAAAAKwnS5kk55AVPP9uST7aWvtkv35iVT0kXaBy1LrW2umLyPdpST7fWjuoXz+oqnbvtz9kGeUFAAAAAFbQUrp4P6qq/nnM9n+uqkcuMruvJblrVd20z+PWSe6Q5FNz0u1RVb+pql9U1X9X1TXnyXe3JJ+bs+2zSW4/LnFVbVlVa2eWJFsv8joAAAAAgCVYyiQ5z0vyuzHbf5Pk+YvM65VJ3p/k2Kr6S5LvJzm4tfb+kTSfTvKwJHdJ8swkuyb536rackq+2yY5Y862M/rt4+yXrlv5zGKyHwAAAAC4DCxlDMrrJzlhzPaTkmy/yLz2TvLwJA9NNwbljkkOrqpTW2vvSpLW2gdG0v+4qr7Tn2uvJB+aknebs15jts04KMnrRta3jiAlAAAAAKx3SwlQ/ibJrZKcOGf7rZOcuci8Xp3kFa21w/r1Y6rq+ulaNL5r3AGttdOq6qQkN5mS7+m5dGvJa+bSrSpn8lyXZN3MelUtrPQAAAAAwLIspYv3YUneUFV3rqrN++Uu6WbdPmyeY+e6YrpZwEddNK1cVXW1JNdLctqUfL+Z5G5ztt09yTcWWT4AAAAAYD1aSgvKF6Tr5v3FJBf22zZLcmgWPwblx5PsX1W/StfFe6ckz0jyjiSpqq2SHJDkg+kCkjsk+fd0Y2B+eCaTqjo0ySmttf36Ta9P8pWqem6Sjya5b5I9003AAwAAAABsIBYdoGytXZBk76p6QboxI89Lckxr7aQlnH/fJC9L8uZ0XbBPTfKWJC/t91+U5JZJ/iXJldMFKb+UZO/W2jkj+WyfkZaYrbVvVNWDkxzY5//L/pijl1BGAAAAAGA9WUoLyiRJa+24JMct5+R9kPFp/TJu/3lJ7rGAfPYYs+2IJEcsp3wAAAAAwPq16DEoq+qIqnremO3Prqr/WZliAQAAAACrwVImydk9ySfHbP9MkjstrzgAAAAAwGqylADlVkkuGLP9L0nWLq84AAAAAMBqspQA5Y+T7D1m+4OT/HR5xQEAAAAAVpOlTJLzsiQfrKobJfnffttdkzwkyT+vVMEAAAAAgE3fogOUrbWPVdX9kjw/yf9Lcl6SHyXZs7X25ZUtHgAAAACwKVtKC8q01j6ZMRPlVNWOrbUfLLdQAAAAAMDqsJQxKC+hqrapqidV1feSfHcFygQAAAAArBJLDlBW1V2q6r1JTkuyb5JPJdllpQoGAAAAAGz6FtXFu6qum2SfJI9OcqUkhyfZIskDW2tm8AYAAAAAFmXBLSir6lNJfprk5ulaTG7XWtt3fRUMAAAAANj0LaYF5d2TvCHJf7bWjltP5QEAAAAAVpHFjEF5xyRbJ/lOVR1dVU+pqmusp3IBAAAAAKvAggOUrbVvttYel+TaSd6S5MFJTunzuFtVbb1+iggAAAAAbKoWPYt3a+3PrbV3tNbukOSWSV6b5HlJflNVH1vpAgIAAAAAm65FByhHtdZ+3lp7TpLrJnnIyhQJAAAAAFgtFjNJzkSttYuSfKRfAAAAAAAWZFktKAEAAAAAlkOAEgAAAAAYjAAlAAAAADAYAUoAAAAAYDAClAAAAADAYAQoAQAAAIDBCFACAAAAAIMRoAQAAAAABiNACQAAAAAMRoASAAAAABiMACUAAAAAMBgBSgAAAABgMAKUAAAAAMBgBCgBAAAAgMEIUAIAAAAAgxGgBAAAAAAGI0AJAAAAAAxGgBIAAAAAGIwAJQAAAAAwGAFKAAAAAGAwApQAAAAAwGAEKAEAAACAwQhQAgAAAACDEaAEAAAAAAYjQAkAAAAADEaAEgAAAAAYjAAlAAAAADAYAUoAAAAAYDAClAAAAADAYAYNUFbVmqo6sKpOqKrzqur4qnpRVW3W79+iql5ZVcdU1blVdWpVHVpV282T7z5V1cYsl79srgwAAAAAWIg1A5//uUmekOSRSX6SZJck70xyVpLXJ7likp2TvCzJD5NcJcnBST7Wp53m7CQ3G93QWjt/5YoOAAAAACzX0AHK3ZJ8tLX2yX79xKp6SPrgY2vtrCR3Gz2gqvZN8q2q2r619qspebfW2unro9AAAAAAwMoYegzKryW5a1XdNEmq6tZJ7pDkU1OO2SZJS/LHefLeqqpOqqqTq+oTVbXTpIRVtWVVrZ1Zkmy9qKsAAAAAAJZk6BaUr0wXcDy2qi5KsnmS/Vtr7x+XuB9D8hVJ3tdaO3tKvscm2SfJMUnWJnlqkq9X1a1ba8eNSb9fkhcv+SoAAAAAgCUZOkC5d5KHJ3loujEod0xycFWd2lp712jCqtoiyWHpWn0+aVqmrbWjkhw1cuzXk3wvyb5J/m3MIQcled3I+tZJTl7ktQAAAAAAizR0gPLVSV7RWjusXz+mqq6frkXjXwOUfXDy8CQ3SHKXeVpPXkpr7eKq+naSm0zYvy7JupHzLeoiAAAAAIClGXoMyismuXjOtosyUq6R4ORNkuzZWjtzsSepLuK4Y5LTllxSAAAAAGDFDd2C8uNJ9q+qX6Xr4r1TkmckeUeSVNWaJEck2TnJvZNsXlXb9sf+vrV2QZ/u0CSntNb269dfnK6L93HpxqD8t3QByidfNpcFAAAAACzE0AHKfZO8LMmbk1wzyalJ3pLkpf3+6ya5T//zD+Yce+ckR/Y/b59LtsS8cpK3Jtk2yVlJvp/kTq21b61k4QEAAACA5Rk0QNlaOyfJ0/pl3P4Tk8w7IGRrbY85609P8vTllg8AAAAAWL+GHoMSAAAAAFjFBCgBAAAAgMEIUAIAAAAAgxGgBAAAAAAGI0AJAAAAAAxGgBIAAAAAGIwAJQAAAAAwGAFKAAAAAGAwApQAAAAAwGAEKAEAAACAwQhQAgAAAACDEaAEAAAAAAYjQAkAAAAADEaAEgAAAAAYjAAlAAAAADAYAUoAAAAAYDAClAAAAADAYAQoAQAAAIDBCFACAAAAAIMRoAQAAAAABiNACQAAAAAMRoASAAAAABiMACUAAAAAMBgBSgAAAABgMGuGLgAAALDx2+F5nxy6CGxATnzFXkMXAYCNiBaUAAAAAMBgBCgBAAAAgMEIUAIAAAAAgxGgBAAAAAAGI0AJAAAAAAxGgBIAAAAAGIwAJQAAAAAwGAFKAAAAAGAwApQAAAAAwGAEKAEAAACAwQhQAgAAAACDEaAEAAAAAAYjQAkAAAAADEaAEgAAAAAYjAAlAAAAADAYAUoAAAAAYDAClAAAAADAYAQoAQAAAIDBCFACAAAAAIMRoAQAAAAABiNACQAAAAAMZtAAZVWtqaoDq+qEqjqvqo6vqhdV1WYjaaqqDqiqU/s0R1bV3y0g7wdW1U+ral3/ev/1ezUAAAAAwGIN3YLyuUmekOQpSf42yXOSPDvJviNpnpPkGX2aXZOcnuTzVbX1pEyrarckH0jy7iS37l8Pr6rbrYdrAAAAAACWaOgA5W5JPtpa+2Rr7cTW2hFJPpdkl6RrPZnkaUle3lr7UGvtx0kemeSKSR46Jd+nJfl8a+2g1tqxrbWDknyx3w4AAAAAbCCGDlB+Lcldq+qmSVJVt05yhySf6vffIMm26YKWSZLW2rokX05y+yn57jZ6TO+zk46pqi2rau3MkmRi60wAAAAAYOWsGfj8r0yyTZJjq+qiJJsn2b+19v5+/7b96xlzjjsjyfWn5LvthGO2HZM2SfZL8uKFFhoAAAAAWBlDt6DcO8nD03XX3jld9+1nVdUj56Rrc9ZrzLa5FnPMQekCpTPLdefJGwAAAABYAUO3oHx1kle01g7r14+pquuna9H4rnQT4iRdy8fTRo67Zi7dQnLU6bl0a8mJx/TdxtfNrHdDXwIAAAAA69vQLSivmOTiOdsuymy5TkgXbLzbzM6qulyS3ZN8Y0q+3xw9pnf3eY4BAAAAAC5jQ7eg/HiS/avqV0l+kmSnJM9I8o4kaa21qjo4yfOr6rgkxyV5fpI/J3nfTCZVdWiSU1pr+/WbXp/kK1X13CQfTXLfJHumm4AHAAAAANhADB2g3DfJy5K8OV0X7FOTvCXJS0fSvCrJFfo0V0lydJK7t9bOGUmzfUZaYrbWvlFVD05yYJ//L5Ps3Vo7ev1dCgAAAACwWIMGKPsg49P6ZVKaluSAfpmUZo8x245IcsTySggAAAAArE9Dj0EJAAAAAKxiApQAAAAAwGAEKAEAAACAwQhQAgAAAACDEaAEAAAAAAYjQAkAAAAADEaAEgAAAAAYjAAlAAAAADAYAUoAAAAAYDAClAAAAADAYAQoAQAAAIDBCFACAAAAAIMRoAQAAAAABiNACQAAAAAMRoASAAAAABiMACUAAAAAMBgBSgAAAABgMAKUAAAAAMBgBCgBAAAAgMEIUAIAAAAAgxGgBAAAAAAGI0AJAAAAAAxGgBIAAAAAGIwAJQAAAAAwGAFKAAAAAGAwApQAAAAAwGAEKAEAAACAwQhQAgAAAACDEaAEAAAAAAYjQAkAAAAADEaAEgAAAAAYjAAlAAAAADAYAUoAAAAAYDAClAAAAADAYAQoAQAAAIDBCFACAAAAAIMRoAQAAAAABiNACQAAAAAMRoASAAAAABiMACUAAAAAMBgBSgAAAABgMAKUAAAAAMBgBCgBAAAAgMEIUAIAAAAAgxGgBAAAAAAGI0AJAAAAAAxm0ABlVZ1YVW3M8qZ+/7h9raqePSXPfSYcc/nL7soAAAAAgIVYM/D5d02y+cj6LZJ8Psn/9OvXnpP+XknenuSD8+R7dpKbjW5orZ2/9GICAAAAAOvDoAHK1tpvR9er6nlJfpnky/3+0+fsv2+SL7XWjp8/60seCwAAAABseDaYMSir6nJJHp7kHa21Nmb/tZLsla4F5Xy2qqqTqurkqvpEVe00z7m3rKq1M0uSrZdyDQAAAADA4mwwAcok90ty5SSHTNj/yCTnJPnQPPkcm2SfJPdJ8pAk5yf5elXdZMox+yU5a2Q5eWFFBgAAAACWY0MKUD4myadba6dO2P/oJO+dbyzJ1tpRrbX3tNZ+2Fr7apIHJflFkn2nHHZQkm1GlusuuvQAAAAAwKINPUlOkqSqrp9kzyQPmLD/jukmvdl7sXm31i6uqm8nmdiCsrW2Lsm6kfMt9jQAAAAAwBJsKC0oH5XkN0k+OWH/Y5J8t7X2w8VmXF20ccckpy25dAAAAADAejF4gLKqNksXoHxXa+3CMfvXJvnnJG+bcPyhVXXQyPqLq+oeVXXDqtox3aQ6Oyb5r/VQfAAAAABgGTaELt57Jtk+yTsm7H9wkkry/gn7t09y8cj6lZO8Ncm26Sa8+X6SO7XWvrUShQUAAAAAVs7gAcrW2ufSBSAn7X9ruoDjpP17zFl/epKnr1T5AAAAAID1Z/Au3gAAAADA6iVACQAAAAAMRoASAAAAABiMACUAAAAAMBgBSgAAAABgMAKUAAAAAMBgBCgBAAAAgMEIUAIAAAAAgxGgBAAAAAAGI0AJAAAAAAxGgBIAAAAAGIwAJQAAAAAwGAFKAAAAAGAwApQAAAAAwGAEKAEAAACAwQhQAgAAAACDEaAEAAAAAAYjQAkAAAAADEaAEgAAAAAYjAAlAAAAADAYAUoAAAAAYDAClAAAAADAYAQoAQAAAIDBrBm6ABuys88+e+girFcXr/vz0EVgA7Kp/77DUqkrmUt9CeOpLxmlroTx1JXMtSnXl4u5tmqtrceibJyq6jpJTh66HAAAAACwkbtua+2UaQkEKMeoqkqyXZJzhi4L693W6YLR1433G2Aa9SXA/NSVAAujvlw9tk5yapsnAKmL9xj9TZsa2WXT0MWikyTntNY23XbVAMukvgSYn7oSYGHUl6vKgt5fk+QAAAAAAIMRoAQAAAAABiNAyWq3LslL+lcAJlNfAsxPXQmwMOpLLsEkOQAAAADAYLSgBAAAAAAGI0AJAAAAAAxGgBIAAAAAGIwAJRu9qmr9csDQZQHY2FXVIX2deuLQZQFgGFW1z8gz9g7LyGeD+Z9SVUf2ZTly6LIAG4eVqjc2lJhFVe0wUpZ9hizLOAKUq1RV7THyi9mq6rAFHDPzgDF2ZqWqOmBOnk9YQJ4nelAANgRVtUVVPbiq3lVVP6uqM6vqL1X1u6r6blX9Z1XtWVVj/3eO1Getqs6pqmvMc77RenifMft3mFOnHrWAaxith3dY6LUDGyf1FsDy+FwMGw4BSmY8qKpuucJ57l9VW65wngArrqrum+TYJO9P8i9J/ibJVZOsSXK1JDsneUKSzyf5WVXtNU+WWyV5zgoX83YLOC+wSqi3ANYLn4thIAKUzKgkL1nhPK+b5PErnOeltNaqXw5Y3+cCNj1VtV+SDye5Yb/pC0n2TXLXJLdJcrckT0ny2SQXJ7lpkpcvIOsnV9W2K1zcl65wfpfSWtunr1N3WN/nApZGvQWw3my0n4thYydASZL8rn+9f1XtvMJ57ldVV1ihPAFWVFU9Ism/p3sY/W2Su7TW7tZae2Nr7X9ba99rrX2htfam1to9k+yY5H/nyXam/rtCkv1WqKgzee5cVfdfoTyBjZB6C2C98bkYBiRASZK8Psm6/ueV+rboVf3rtkmetEJ5AqyYqtouyX/2q39Oskdr7UvTjmmtHZOuZdJrpiT7SZJP9T8/vqqus9yyJjkkyW/6n19SVbUCeQIbGfUWwHrlczEMSICSJDk5yVv7n+9dVbdbgTyPSPLD/ufnVtVWK5DnWCODDx8wZt8lZiCsqstV1TOq6jtVdVZV/b66mbn2mnPc1lX1nKr6flWdXVV/rKrPV9Vd5ynLtavqSVV1RFUdV1XnVtW6qjqlqj5aVXtPGqh+Tj5bVNVTq+rb/aD1f+zL/PT+GhY8+1ZV3a2q3lNVJ1TVef31/LCqXlVV156vLLAJe3qSK/U/v7i19tOFHNRau7i19p55kr2of90yyf5LLN+oc5O8sv/5lkn+eQXyHKvmmXF1bp1bVXeuqo9U1al9HfOzqnphVV1pznH/WFWfGkn306rar6ouN6Usl6uqf6qqN/b14R+qmwDkzKo6uh+E/uoLvK47VtWHquqMqjq/qo6vqv+qqhv3+xc0S2NVXbeqDqqq7/XlOb+qflVVH6iqOy+kLLAM6q0Jquo2VfX2qvpF//x1flX9urrJgt5UVfeZFiTtn61e2ac/sz/+hKr6UlU9s6q2H3PM3PrwLlX1P/15/zK3Hq2qW1TVC6rqs1V1cv+M+Kf+mfFdVfX381zjXyfe6NfX9tuO6fM5o69nbz/nuGtW1YFV9ZP+3pzZP5futIj7u2VVPauv+87qnyePrqonV9XmC81nSv5XrKqn9ff7jKq6oKp+U1Wfq6pHLeQcVbVbdc/gp4+8f2+tqpstt3ysGhv75+LNq/v8+9n+7+CC6j5HHldVX6yq51fVzefJ4x+r++x4fF9fnNXXHYdV1QNrTivQuvTn7S37v+Wjqpuw7RKf02sFnu1qdkKhQ/r1navqvX3de15V/V9VvW5uPlV1+76O/lVfR/yyr/e3XsQ9vllfr5zQ53Fan+duC81jAfm/ob/nZ/XXc3xVvbMW0Kq3/x14cn8vz+7z+F5ff2/446C21iyrcEmyR5LWL/skuXa6b+Jbks9OOOaQmWMm7D9gJM8dktxvZH2/Ccec2O8/chnXMnOOA8bs22dk/62SHDWyPnd5Rn/M9kl+PCHNxUkePqEcmye5aEr+M8vnkmw15XqunOToKccfna671l/fvwn5XCnJh+YpyzlJ7j3076PFclkv6bpG/qb/O/hTkrUrkOcl6rMkH+nX1yW5/pj0l6iHx+zfYbR+S9f18tR+/adJNhtzzCXq4SVexyH98SdO2D9apuf19eK4+uXr6SbdqCQHT6mHPp1k83nKMm35XZJ/mOea9p9SzrOT3D3JkaPv34R8HpPZ/5WTlrclWTP077hl01vUW1Ov4+lZ2DPY2OevJM9KcsE8xx455rjRa335mGNOnHDvpi0HTbnO0Xt1vSQ/n5DHhUn+uT/mVumCLuPSnZ9uiIBx59pnJN1OSb4zpcxfTbL1hHwOmXsvxqTZdUoZZ5ajk1xrSh7PnPI78Kck98oC6nnL6luyiXwuTvfM9ZUF1DFHTDj+aunGM57v+H3mHDdaV+yS5Ptjjjlg3L2bskx9thu5V4ckeUS6/1nj8vl5km37Y56Vyc+C383k/w9/rTf6euRPE/K4KH08YUI+l7oXY9K8MMlfptyXi5O8ZMrxWyf52pTjv5OuPh/7Xm4IixaUJElaa6dltsvQ3avqDiuQ50fS/bEnybOqau1y81ymt6YbOP7N6bo67ZLksUlO6/e/uqpukS6od8Mkr0j3D2vXJE9Lcla6Dwdvrqprjsl/5lv5/03y7CT37M+3R5JHJ/lmv/9uSd40pZyHJblt//M3kzykL+u9kry33/df0y60/5b540nun67yeX+6lgu7JNktyVOT/CrdP7IPVtVtpuUHm6CbJ7lG//NXW2tnr4dzvCjd39/l0j1wLEtr7bwkB/Wrf5vkocvNc5nula48R/Vl2SVdvffpfv/t0wUwn56uzvl0kgemqxfv2x+X/pjHTTjHmiTHJ3ltkr3T1V+7Jvl/6erBC9I9UH94Qr2cqnpIkgPT1dF/6Mt0+355broP8oel+0AyUVU9Ol3w8QrpvsTaN8kd0s2U/MDMdo99TGZbjcFKUm+NUVW3Std9fbMkJ6QLVN013YewO6Z7Bnt3ug+V445/YZJXJ9kiyR/Tje95t3R/23dJ96H26+nuyyT3T/L8JMf057ttkt2TvG4kzZp0rUoPTze7+h79Oe7Zl/mkPt3zqupRC7j0/0k38cZB/bl2TVffnp3uS/O3V9UNknwiXb21f7o663ZJXpyu/twyyTtrSkv23lvS1d0fSPKP6er7hyb5dr//DumeURetutmSv5TkOukC8C9Jsme69+8e6Z6ZL0x3Tz9aVVuMyeOBmf0dOCvdezFTz78gXeDgfUm2W0oZWV024s/FB6Sr85Lu7/4hSf4h3d/uPdM983w1Y+qyqrpiur/Dmd6C3003qc8/pPt7v3+S/0j3hdM0b09y6ySHJtmrP/f9033BMGPZz3Yjbp3u2ez/0tW9u6art2d6DNw0yWuqGwf51X05HpbZZ9aZZ7ed09UV02yXrh65MJesY/ZPV+9uluS1VfWAefIZq6pemm5SuTVJvpEuTrFbX9aHpYsLVJIXVdW+E7J5b7r3LEm+ldk4wl7p/mfcJl19vuEaOkJqGWbJmG/A0z30znwj8L9jjjlk5pgJeR4wkucO/bZ/HNn2ojHHnJhlfpM5kv8BY/btM7L/4iT3G5Pmlukqmpbuwej8JLcbk270Wp4+Zn8lufE8ZX3JSFluMmb//UfO8ZGMaVWU7iF24rdYc9JckOReE8pylcy2FP3q0L+TFstluaT7YDXzN/TyFcrzUvVZug+iLd23oTeak/5S9fCc/TvMrd/SfZj8db/tF3PriHH18BKuY6auP3HC/tH654gxZdg83UNUS/fAdl6S/xiTzxVH7tkPJ5zrRklqSllvma4leEvysjH7t0xyRr//zCQ3HZPmpv2+mWs6ckya66ULLLT+/oxtIZnZFlQXjTuXxbKcRb018Rpe2h/7p0xvYbdN5rTgTPehdKbV3c+TXHfK8ZfaN6c+/EKSLaccf/UkV56y/3Lpetm0/n0Z9ww4eq8W8rz6m3QTKd1oTLonjaS7/5j9+8y5vku1+kr3QfozI2n2GpPmkJlrGrOv0nV9bUl+kOTqE+7NPUfep8eMuW8zrXT/mORvxxx/i3SBy4n1vGX1LuPqtWyEn4vTNT5pSf5nnnRXHbPtP0bK9sZMePbq/96uNWfb3Lri0fOcf1nPdnPuVUv3BdIVx6SZ+V92YbrnvPmeWX+XMc93mW1BOa2O+buROuaUJJcbk+YS/xvn7Nt1pI6bdM2bpfuybeb5+spz9v/TyDk+OeFaXpRLvlf7XJZ/awtZtKDkr1prv01XISXJnWsFxtJqrX0qs61knlFVV1lunstweOu+vbqE1g0e/7V+9RrpPkgfPSbdpzL77fYdx+xvrbX/m6cML01X+VWS+4zZ//j+9fwkj2+tXTQmzeuSfG/SCfpvlp/Zr76xtfbpcelaa39I19IzSe5Q/ThssEqMjklzxno8zwHpvpBYk9nx3ZastbYuXRAsSW6S5F+Wm+cy/DnJv86tp/r1mW9nt0734fg5cw9urf05ybv61VtV1TZj0vyy9U9U4/T199v61fuNSXL/JDPfvr+ktfaLMXn8IvMPhP/UdAHVU5M8obV24YR0L073YLpZhn1v2DSpt8bbtn/9RWtt4n1prZ3VWrt4zuZnp/t7bUke3Fo7ecrxE/elu1+P7a910vG/a639ccr+CzL7XHb9dMP5THPwAp5Xr5HkBa21X445/p3pnjeTMc+1c/woY1qG93XhY9MFs5PFTwCyV7ou6EnyL621341L1Fr7TLrgQpLMbV16v8y2gH9Za+1nY47/cWZ/B2FeG+nn4pm68KvzlOP3o+t9Of61X/1ekqdOevZqrV0wrZ5NF8x9xzznX+6z3VyP7Z8p53pz/7p5kstn8jPrzJijV0vXU2GaSXXMTzJbx2yXrqfQYjw33f+i72bC/93+/9e+6bqzb52uxemoJ/av65I8bsKz6oHpGihtsAQomevV6b6xSJKXrVCeM39k22Q2cDaEw6bs+9HIzx9YQLobzneyqtqsqrbrB7q9Rd99/G/TjbGTdE3SR9OvSXKnfvUzkyr/vkJ/95RT3zazD2qHz1PMr4z8vCID+8JGYnQw7HPX10laN4HFTN3zsFqZgfrfnu6b4yR54bjubpeRz899yB0xWqd+qLX2lwnpfjjy8w3mO2FVXaWqblRVfzdSr/6x333zMfdipqvSxZleb74nmdp9c+ZB8+OttfMnJeofBmeG81CnstLUW+PNDNVz86q67dSUI6qbtPCe/eqXW2vfX0YZvt5aO3ExB/QTSWxfVTcfqc9GJ/G59aRjewt5rm2Z8CzYuu73x/Wr8z3XvmtMcHcmn5PTtfxMkj1qcRPmzNStP2+t/Whqytln1l3nnGPPmaJk9kuvcd6Z6fU8zLWxfS6eqQv37rtsL9Sd030JmyRvmNBAZqEWPdTDEp7tRv1wXMCwN1qnTHtmHX0WnVYXLqaO2XNKukvor+9e/eoR8wRv/5huKJFk5DmzjyPs3q9+rrU2tit+X49Pu4bBCVByCa21M9NNZpAk/1BV91iBPD+f2W9ynlpVV1tunkt0qZYzI/64yHRjZ/qqzsOr6kvpugWckuTYdBXJzLJjn3zu7GQ3SjdGUDI7Rskk35myb5eRn79Zs7OqXWrJJcdj2jawepwz8vOVJqZaGS9J121j83Qtk5alD/bNPCjfIN2YO0NYyTo1mVyv3rKq3lFVpyX5fbpxhn6c2Tr1gD7pZumGrhh1i/71hL7V+Fj9Q+vxE86/TZKZFuaPn1an9vXqzDfa6lRWmnprvPena8G3ZZKvV9XHq+oJ/YfdibN29+W4cv/zV6akW4j5gmtJkqq6UlXtV1U/TBdkPinJTzJbn40GSafOYpuF1a2/m1b3ZZ7n2hHfnmf/t/rXK2YBX+KPmHlmvdkC6taZ1myXS3LVkTxu2b+eMKkFZvLXFnEnLqJsrHIb4efimcDT7ZOcUN0s2fevqmtMOyjdeK8zLqu6cDnPdqMuk2fR3mLqmFtMSjfGzTMbID5oAXXhTL05+px5o5E8Flpfb5AEKBnndZn9Q33pCuU5M9D6VhnT1e8yMq7p94y/fis8oYn43HSX+na4qi6fbryHd6cby+QKc9PMMXf/aOX7m3mO/e2UffMNJjzJYr5pg43d6APGtdbnifouxDOt9x5UVX+3Atkemu5hLkn2r6otVyDPxVpQnbqIdOPq1cek6270qCws4DepXp2vTk0m16vqVDYU6q0xWmvHppsI4A/puqXfO90EFz9O8puqendVjevCPBoAPG3M/sWYFgRMklTVDuk+eP97um7N87U0nPocucDn1WlpRtPNV5b56tDRXj9XnZjq0laifl1MPb8+h0Zg07QxfS5+WZJ3pGvFd80kT043+esZVXVMVb2kqsb977is68LlPtuNmljHzWn1veRn0RGLqWOGqgeTxdXXGxwBSi6lbzo8M+vgbavq3iuQ55fTzW6dJE+ZUDlu7PbPbPPsLyd5ULpWN1ulG5C3WmuV2W/Npn2rvxyjFese6b5ZXsjyn4HVY7Q7x86Xwflemq6Fz2aZf7zDefVdiWcelK+XybNgb7Sq6m/Szea4Jt3D1rPTzT54tXSDj8/UqY8ZPWw9FGW0Tj04C69Tl93SAuZQb03O+4PpWkQ+Pt0H8pkvHK6e5OFJvlJVh/TdusdmscwiLKRL5Lv7MrZ0QYS7p7sPlx+pz0brm/X1nLgU892fpZZ15nq/noXXrbfMJWcSnjn3Qt7DDemeshHYmD4Xt9b+0lp7TLrWewemmwn6gnS/97dI1738/6pqseMjLsbUunADerZbivVVx4zW+8/OwuvB0Z4Io+ddX/X1ZWLN0AVgg3VwukkBrpbkpVX1yRXI84VJ7pIu2v+8JE9fgTw3CH0Xosf2q19LcpdJY/VkcjP10W+c5vsmZVpT/TNHfr6gHxgcuKSfpmuNdPUkd6yqta21s9fXyVprJ1TVIek+kD+gqnZcgWzfm+T5Sf4myfOr6m3zpN/Y7JPuOeWiJHtMGWNoWtefmXp1Id9OT6pXR+vUK6pTGZB6a4rW2lnpJjt4a5JU1c3TTUi4b7pJCx6Zrgv16/tDRlukbrdS5Rin/1B+h371oNba/hOSDjmZ5DTXyvQukqN17KRx3sY5s8/7GsuoW2fOt5Agz1JbKrG6HZyN6HNxP47wC9ON93uFJP+Q5KHpJijbKsn7q+pGrbWZ1pKjdeG1k5ywUmUZY58s/9luKIupYxZbD87YYol14ej55ivnBl0PakHJWK21c9INDJx041LcbwXy/EaSz/SrT6iq9foweBm7amabqB8+KThZVVslmTTY/C8zO5viLhPSZAH7R8cvuvs8+cCq1A9AfUi/eqXMfsGwPh2Y2W+yV6I10sWZHaPn2pmdvW9TMdOldNoA6Mn0+vAn/esNqmpid5t+39hx0/oxhU7pV/ecZ0w7WG/UW4s+109ba69I8veZnVToQSNJTsjslxh3yvo12kV+2uQ28z3/DWXXBe7/cyaM5zvBzDPrTavq+osuVWdmwogbTBvPrx+Hb4clnoNVbGP+XNxaO6+19oXW2qPTtc5Lui7Toy1Bvzfy82VVFy7n2W4oi6ljFhNk/Em6/7PJ0j+7/zKz3dgXWl9vkAQomeaNmR3D4CVZmebAMzOXXT7dN+ibitHWyNPGHXtMkrEzkfVdn2YGJr7HpOb+/YfjR0w5x9cy+y3KE6pq7ZS0sJodnNl/5i/tW7jMq6o2q6qHL/ZkrbVfJfnvfvU+WZkHhMMz++HseVn/E2dclmbq1Yl1alVtm9lZYMf5Yv+6WbpunpM8PNP/x32sf71hZifBgSEcHPXWorTWfp3Z1n9XH9l+cZJP9au7V9VOc49dQQt9TnzCeizDcjxi0pczVXWdzH6oPnKRMwB/bOTnpY7F94WZoqRrITbJPtnAuzayQdsUPhd/ceTn0XEnv5TZL3H2rar5xqRdjpV4thvKYuqYL0xJdwn9eMIz780eVXXbxRasjyN8uV+9e1Vde1y6fpiTRy42/8uSACUTtdbOTfLKfvWWSf5xBfL8dpKP96uPy4bZfHspfpvZAZQfXFWXm5ugqnZN1xJhmrf0r5dP8pYJ/yCekSljT7XWzk/ymn512ySHVdXEh/+q2rqqnjJPuWCT01o7JcnM7/6Vkny5qnafdkzfZfCzSZ61xNP+e2ZbSi97YPS+RdUB/eo10z0cbSqO619vWlV/P3dnVV0xyfsyffD0D2f2A8WLq+omY/K5SZIXz1OWVydZ1//8X1U19Zv9qvrHqrrVPHnCoqm3Lq2q7ldVV56y/3rpupQnl+66+Jp0EyRUuuel607JZ+K+BThu5OexHw6r6olZgZZZ68mOmW199VdVtSZdAHvmuXex45l/MMlMK6on9pNnTFRVt6iqf5qz+SOZndjjhVV1qZ5K/d/ApG71MK8N/XNxVV21qu4zTy+P0dZ5f60L+3E2Zz6D3ibJwVO+kNiiqpbTRXglnu2GNKmO+dvM1jGnJfnoIvN9eWbHjjysqm40KWFVbV5VDx3zP2mm/t0yk+MI+6X7/d1gCVAyn//M7D/9q09LuAgvSvcHeLkkm0Trvv5b+Pf2qzsm+WpVPbiqdqmqu1bVa9O1jjw/U8bwaa19KMnn+tX79vk8qKp2rqp7VNW70z1Mf2v0sDFZvSqz38TcK8lPq2q/qtqjqnasqjtW1WOr6j3p3t8DlnLdsLFrrb0zs99gXzPJkVX12ap6UlXduap26v+Gn1hVn0jyoyR7LuN8p6YbHDxZuTr1w5ntJrdSeW4IZmYQ3izJp6rqeVV1p6q6bf9B/gdJ7pxucoWx+i9sntavXjXJ0VX1nKr6+355TpKj+nPMPDRfqk5trZ2Q2ZZNV03y9ap6Wx8Y2bkv0wOq6hVV9X9JPplk+2VcO0yk3rqUpyU5paoOr6onVNXu/bPOnavq2enqiJkPu5cIoLXWfpDZLyhumuSYqjqwv3879s9NT6uqr2S2TlqK72e2y98Tq+p9VbVXX3/ct6r+J8mbM6U+G9h3kryyL/c9+3Lvna68MxNEfry19onFZNq3ttw7yZ/SBYnfVlWfqap/qarb9ee5Z/8M+/V0LW93n5PHBenGGU26AM9R/f+Lv6+q3apqv3SThSSXDBTDYm3In4vXpguKHV9Vr+0/P96uqm5TVfeuqrdkNsB6cmYDozNemNmW7U9J8u2qelz/d7RzH/x8VbrA5nKCs8t+thvQcenKPVrH/H1VPS/JN5Ns06fbt6+XFqy19vXMTiJ3gyQ/qKqDq/vCe6f+PA+uqtcn+VW6uMOV5+Tx8cy+r/+U7ll175F69LB0jaW+vegrvyy11iyrcEk3u3Prl33mSfuUkbQt/ZffY9IdMJJmh3nyPGJOnkcu41pm8jhgzL59FlKm0bLPc65D+nQnjtm3TboH0DZhOTPduB5HTrvmdJXN0VPy+V66b7dm1veekM8VkrxrSj6jy/FD/05aLEMuSR6Q7qFrIX8vP05y9zF5nLiQ+izd4NXnzslznzHpdphWv81Je+8x5dxhifdiYj3X75+3THPKfqlrG0m3x0i6Pcbsf9GY6xpdXrOQej7dt9oXT8jj3HQP21/p1z89pbx7JzlrAb8jFyW589C/15ZNe1Fv/TWfIxdw/Rcm2W9KHvulm7F8Wh6XukcLvdY+7Y7phuCZlP+P0o3LOTHPrMDz6oR7N+7a9hkpy07pnj8nlf1rSbZealmS3CrdF/gL+V1+0YQ8npWu7p1Wz0+8XsvqXbIJfC7OJeveacvJSXaakMfV03UTni+PfeYcN1pXTL3WPv2yn+0y+7/rkHnONW8dnXmeWUfrjSR75dL/C2eWi5I8c5lleVq6Bk3zvQfrktx4zPFbp6uPJx333XT1+YJ+34dYtKBkIf47ya9XOM8Xp/uwuMlo3eyR/5DZb6DOT/eN8M/SVbS3bq19ZXIOf83nj+lmenxaukrkT0nOSfeN0n5Jbp+uApxx1oR8zmutPTLdIMP/mW4A3rPSPaT/sc/v7enGU/vbBV8obIJa13r5ZkkeluQ9SX6ebvKEC9N9oPxeutYtd01yy9ba5yZktZBznZFuLKMV07pWK9+aN+FGprX20nQPg59L935ckO7h+kPpgi0L6rLaWnt5ulY3H0nX5XtdkpOSvCPJLq21T2W25cLYOrXP5wPpHmSfl+5B9TfpghozE0N8PN0wHDu01r604AuFJVBv/dWD0t2DQ9I925ye7h78KV1g9s3pPpAfNKUsByW5eboxPn+c5Ox0z3HHp+uR8rQkD15OIVvXWnPHdK1RT0pXd/w+3T14VpLbttlZdTc0f0j3/Llfunt8Trr7++10rRd3b91EIkvSWvtRuvv/yHT19K/T3f8L0rVYOzJdy5/b9P8XxuXxmiR3TPf/YVI9D8u1oX4uPimzQzF8Ot3/gz+mqwt/ly7w+Kwkf9ta+/64DFprv2ut7Z7uy68j0j1vrUv39//jdK327puuC/aSrdSz3RBaa59M99n6nenu+QXp6psPJrlDa+21y8z/4CQ3SvKydD18fpfuPTw33Zc4H0zXo+c6rbX/G3P8OekC7vumq5/nxhH+IbOTw22Qqo+0AhuR6ga6n2kif+PW2i+HLA/AxqyqtkgXmLxCkgNbay8cuEgAALCqaEEJG6eH9K+/TfftPgBLd7/MjlF31IDlAACAVUmAEjYwVXWdqpo4c1l1MxzODE58aNMMGmCqqrrxlH07JHldv3pGutmOAQCAy9CaoQsAXMrdkryqn2nryHTjW2yWbjyKvdO19Em6D9KvGKB8ABubY6vqU0k+kW483nPTzX5853Rj+Vy5T/es1tqFg5QQAABWMWNQwgamqvZJN/DuNKcl2WvSIMcAzKqq+R52Lk7ygmmTaAAAAOuPACVsYKrq6ulm1r5nutm1r5Fk63Qzsf0s3Syx/7Wc2RIBVpOquneSe6WbhfZaSa6WbmbKU9K1VH9Ta+3HgxUQAABWOQFKAAAAAGAwJskBAAAAAAYjQAkAAAAADEaAEgAAAAAYjAAlAAAAADAYAUoAAAAAYDAClAAAAADAYAQoAQAAAIDBCFACAAAAAIMRoAQAAAAABvP/A9x5LF3p0D9BAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 1600x600 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KfGPkFAL--_f"
      },
      "source": [
        "### Checking Model Parameters"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "V1-hB4bv--_f"
      },
      "source": [
        "# print(dir(model_cnn))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bMOcHWwb--_f"
      },
      "source": [
        "# print(dir(model_fnn))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hJv51VJR--_f"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "kOyJBCa0--_f"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}