5.2-Keras-and-digits.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# High-level interface for neural networks\n",
    "\n",
    "Keras provides a high-level interface for neural networks. Other high-level libraries have very similar interface.\n",
    "\n",
    "Keras can use both theano and tensorflow as a backend for computations. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "env: KERAS_BACKEND=theano\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using Theano backend.\n"
     ]
    }
   ],
   "source": [
    "%env KERAS_BACKEND=theano\n",
    "import keras"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "from matplotlib import pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import numpy\n",
    "from keras.datasets import mnist\n",
    "from keras.models import Sequential\n",
    "from keras.layers.core import Dense, Dropout, Activation\n",
    "from keras.optimizers import RMSprop\n",
    "from keras.utils import np_utils"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# the data, shuffled and split between train and test sets\n",
    "(X_train, y_train), (X_test, y_test) = mnist.load_data()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(28, 28)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train[0].shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAV0AAAD/CAYAAABSKwXmAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsnWlwXNd153+9791oNPZ931eCIAiu4GpSJCXKluRoSew4\n5aRS/pBUxamZmsl8meRDUpWpqfIkLpeTclKuJDZlyZZskeIi0tzBHcS+NvYGGg00lsbSjV7nA6uf\nSYmSLAlANxrvV6UPaj52n3d53/+de+4550pCoRAiIiIiIhuDNNIGiIiIiGwlRNEVERER2UBE0RUR\nERHZQETRFREREdlARNEVERER2UBE0RURERHZQETRFREREdlARNEVERER2UBE0RURERHZQETRFRER\nEdlA5Ov9AxKJJGrrjEOhkCQSvxvNYwKRGRdxTJ5PNI+LOCbP5/PGRfR0RURERDYQUXRFRERENhBR\ndEVEREQ2kHWP6YpsDHq9nqSkJPLy8sjKyiIUCjE2Nsbg4CAOh4OlpaVImygiIsImEl2p9FmnXK/X\no1KpkEgkKBQKlpaW8Pl8xMXFodPpkMvlBAIBXC4XCwsLuN3uCFm+voTvv6CggMbGRpqamti5cyeh\nUIi7d+9y5coVbt26RUdHR6RNjSrkcjk6nQ6LxYLL5WJ+fh6/3x9pszYciUSCyWTCZDKh1WoBmJ6e\nxuVy4fV6I2xdbLIpRFcmk6HVap8R3v3791NQUIBarSY1NZVr164xMjLC66+/TmNjI0lJSczNzfHB\nBx/w/vvv09LSEsE7WD8UCgVZWVm88cYbfOMb3yA+Ph6NRgPA0aNHqaurw2g0iqL7MZKSktixYwd/\n+qd/yocffsg777yD3W5nqzX112g0HDt2jJdffpn6+nqkUin/+I//yNmzZxkaGoq0eTFJVImuwWBA\nqVSi0WhITU0lNTUVs9mMTqejpKQEvV4vXJuZmUlcXJwgxBqNhrm5OY4dO0ZycjKBQIDp6WmcTicr\nKyuRuqV1xWg0UlBQwDe/+U0OHz5Meno6MpkMn8+H3+9HIpGQkpJCTk4OGRkZTE9P4/V6N42wGAwG\nUlJSUKvVOJ1OJiYm1uy74+PjycvLQyaTIZFIkMuj6lHYEBITE6mtreXll19m586dpKenI5FIMBqN\nW3I8NoqoGFm5XI7RaGTXrl2kpKSg1+vJyMggKysLi8WCTqejtLT0GdEFCIVCrK6u0tLSgkKhID4+\nntnZWSYmJpiZmcFqtdLS0sLMzEyE7mztkUgkyGQydDodFRUVHD58mJdeeomMjAzhQVleXmZqaoqp\nqSmqqqooKyvjxRdf5P79+1itVmZnZyN8F78fcXFxVFVVYTab6erqWlPRNZlMpKam4vP58Pl8a/a9\nmwmLxUJNTQ3btm0jOTl507yM1wKDwYDZbMZisWA0GtHpdCiVSgDGxsZwOp34/X4MBgMejwen08nS\n0hLBYPAr/3ZUiK5Op6OwsJC/+qu/or6+Xlgef5xQKCTctFQqJRQKsbKywm9+8xsGBgaEh2dubg6X\ny4XL5cJms8VUbEoul2MwGCgoKOCll17izTffxGKxPOOZLCws8PDhQz766CO+//3vs2fPHsrKyviP\n//gPzpw5w7179yJ4B78/cXFx1NTUkJyczPLyMrdv316T75VKpRgMBlJTU0lISECj0WwpwYEnL2+d\nTkdKSgoajQapVEogECAQCOD3+9dEXKKR8B5Ibm4uDQ0N7Nq1i/LycrKzszGbzUgkEs6cOcOdO3dY\nXFyktLQUm83GzZs36e7uxuPxfGUbokJ0V1dXcTgcLCwssLq6+qmiOzMzg91ux+PxkJ2djclkYnFx\nkbt379LZ2SlMFL/fL0yeWPNiiouLefXVV9m5cydFRUWYzWZkMtkz1yQnJ5OWloZSqWR4eBiz2Sys\nHkwmU4Qs/+KkpKTQ1NSE3W5fs++USCQYDAbKy8vZuXMnvb29DA8PMz09vaWENycnhz179vC1r30N\nk8nE/Pw8ra2t/OQnP+Hx48dMTU1F2sR1ISkpidraWl577TWqq6tJTk5Gr9fjdDqZnJxEp9OxY8cO\n6urqCAQC6HQ6BgYG0Gg0jI2NxY7o+nw+ZmdnaW5uRiaTkZyczPz8PKWlpaSlpREMBpmfn+fatWtc\nv36dlZUV8vPzKSwsRK/XY7PZmJ6ejvRtrDsmk4nS0lKOHTtGUVERBoMBn8+H3W4XhKmqqgqJRMLq\n6ipTU1PcuXMHg8HAtm3b0Gq1KBSKCN/F749GoyEpKWlNwyFyuZw9e/awY8cOzGYz09PTzM7Osrq6\numa/Ec0olUoSEhI4evQoR44cISsrC6VSSWdnJ1euXOHSpUssLS3F1OoQnrxs4+Li2LZtG2+++SaN\njY2C2IadNrvdjlar5dChQ1RXV5OUlAQ8cfY+7th8FaJCdAOBAIuLi1y8eBG73U5qaiqjo6O88cYb\nGAwG1Go1fX19nDt3jp/97Gd4vV6Sk5OpqqqioqICl8sV6VtYV6RSKUqlkoqKCnbu3ElJSQlqtRqv\n18vMzAy3b9+mp6cHtVqNxWJhZmaGtrY2+vv7cTqdZGZmUl9fj8FgQK/Xo1Qqo/6hUigUqNVqFAoF\nEsnalfgrFAoOHDhAdXU1Pp+P8fFxFhYW1uz7ox2dTkd1dTUnT55k9+7dqFQq3G43jx8/5urVq8zO\nzsZcaEGhUGAymaiurub48eO89NJLBAIBent7uXnzJufOnaOrq4v5+Xni4uJISEggJyeHxMREAoEA\ndrudvr6+NXtmokJ0AYLBIB0dHQwPD6NSqVhaWsJsNmM0GqmpqeG3v/0t7e3two1PT09z/fp17ty5\nw/LycoStX180Gg1ZWVn8yZ/8CcePH0ej0SCRSLDZbNy4cYMf/OAHQq6u1+vl9u3btLe3MzU1xdDQ\nEHv37kWlUgkvqfv37zMyMhLVD1dSUhLp6enExcWt+U66Wq1GqVTi8Xjo6urC4XCs6fdHMyaTiaam\nJgoKCtDr9fj9fgYGBrhz5w4PHz6M6jnxZTGbzezdu5c/+ZM/obGxEblcTm9vL++88w7/+Z//ydzc\nHKurq1gsFhoaGqivrycnJ4dQKMTCwgLd3d3cvn17zbKgokZ04UmYYWFhAalUit/vZ35+nqWlJWQy\nmRBqePToEfBEpFdXV2N+WWgwGMjPz+fw4cNUVlZiMBhYXl5meXmZtrY2bt26xejoKH6/H7vdzuPH\nj7Hb7SwsLAjJ/n6/H6lUik6no6amhsbGRsbHx6P6AcvNzSU/Px+VSvWJwpgvS9jjSUhIQK/Xs7i4\nyMzMTMymFH4cpVJJamoqhw4dIiUlBb/fz+zsLBcvXqSjoyPqVz9fhsrKSg4cOMCLL75IeXk5y8vL\n3Lx5k7Nnz3Lz5k0cDofwnCQmJnL69Gny8vJQKBSsrq5y4cIFrl+/jtPpJBAIrIlNUSW68ERMw2Iw\nMjJCT08P1dXVlJWVUVpaSnNzM7Ozs1ti00MqlZKfn8+BAwc4fvw4aWlpTE9PC8La0dHB48ePWVpa\nwu12Mzc395nfJ5PJyM7OpqSkRMjnjVbS09PJyMhAJpOxsrKyJhsYCQkJNDQ0kJmZiUKhEKoVY/3F\nDU+8++LiYpqamiguLkatVjM9PU1bWxtXr16NuUIIpVJJdnY2R48e5dSpUzQ0NLC4uMjDhw95++23\nuX79OhMTE4JDYjQayc/PZ+fOnVgsFhYWFhgYGOD8+fM8fvx4TV9IUSe6T3Pv3j20Wi3l5eVs376d\nmpoaKioqePToEV6vV8hSiEUkEglKpZLGxka++c1vUlNTI6RN/fM//zNWqxWPx4NEIvlCE0Kn02Ey\nmdY0TroexMXFERcXRzAYxG63f+nNtHBes1wup7S0lLfeeov8/Hy8Xi/j4+MsLS3FfPmvVColISGB\nEydO8K1vfQuVSkUwGGR4eJizZ8/S0tKC0+mMtJlritFo5OTJk7zyyivU1dWxurpKV1cXFy5c4Be/\n+AVer5dgMIhcLkev11NaWkpdXR1JSUkoFAp6e3t5++23uXnzJuPj42tqW1SLrs/no6Ojgx/96Ef8\nr//1v6irq+Mv//IvaW1tZWRkhPb2dh48eBBpM9eFcHlvZWUlxcXFyGQy3n33Xd59911aWlrweDzC\niiBWXzzwZOUzNjb2pQpclEolFouF4uJiamtraWxsZNeuXZhMJu7evcuvf/1rpqamotrjXwvCL5yy\nsjLS0tKQSqVYrVZu3rzJhx9+GHMbiQqFgqSkJA4dOkRubi4ul4uuri7+9V//lcuXL7O6ukooFEKv\n15Ofn89bb71FbW0teXl5aLVaent7uXTpEu+99966ZEVFteiGQiFmZma4d+8eZ8+e5eDBg9TV1ZGT\nk4PT6eTevXvIZDIGBweZn5+PqYfHYDBw9OhRqqqqUCgUjI2N0dzczMOHD7/wQyKRSJBIJJs2JPN0\nyCmMQqEQktn1ej3JyclotVrkcjkajYacnBx0Oh06nY6kpCSMRiNZWVmYTCZkMhl2u522tjaWl5c3\n7bj8PoT3BF544QWqq6uRSCSMj4/z4YcfcuHCBcbGxmLquYHf9WrJzMzEZDIxNDTEO++8Q0tLCzKZ\njD179pCenk56ejoFBQXs27eP9PR0dDodAF1dXdy/f1/YK1lrolp0AbxeL1NTU/zqV78C4MSJE2Rl\nZVFaWkpqaipyuZxLly7R1dXF1NRUTCwVVSoVaWlpHDt2jMLCQpaWloQUsC9b0hwOJ/h8PjweT9QL\nTSgUIhQKIZVKycnJoby8/JlcSb1eT25uLlKpFIvFQn5+PkajEZlMhkqloqysDJlMhtvtZmFhgeHh\nYSYnJykqKkImkzE3NxeTgvM0crmcnJwcXnzxRY4fP052djbLy8tYrVYuXbrEnTt3YnLzLBxSUigU\nyGQyVldXsdvtZGVlkZqayvbt2ykvLycjI4P4+HhUKhUymYxQKITf76erq4vOzs51mxtRL7rwZPe9\nra0Np9NJe3s73/72t6mtraWgoIBvf/vbJCUl8atf/YobN25smr4Cn0V4YhQWFmI0GhkdHeX+/ftf\nWnCfLp+emZlhcnIy6kXX7/fj9/tRKBR8/etfZ8+ePc/82+p0OkF0n36hOJ1OxsbGuHv3LoODgwwM\nDNDV1YXT6eTw4cPs2rULlUqFy+XCbrdHdQbHV8VoNLJ7926+//3vC8Li9/uZmppiZmYmZlMtg8Eg\nXq+XxcVFVldXKS0t5Uc/+pHwEpfL5cjlctxuN0tLSywvLwvNthYXF+nu7sZqta6bfZtCdAEhJer2\n7du4XC62b99OU1MTe/fuZf/+/UJv1J/+9Keb3ttVKBRotVrUajV+v5+JiQlu3br1heJLEokEtVrN\nrl27qKmpwev1MjIywqVLl/joo4+i3sO7evUqPp+PmZkZkpOTkclkz3hlwWAQq9XK8PAw4+PjzMzM\nMD4+ztzcHMvLyywuLrK4uCj0yi0sLKSoqAidTic8aLEsuAB1dXU0NDQIZfWhUAin08nZs2fXfHMo\nmvD5fNhsNv7lX/6FV199ld27d2M0GllYWGB8fJz+/n46OzuZnJxEoVBw6tQpysrKcLvd3L17F7vd\nvq77JJtGdOFJj4aJiQmcTiejo6OoVCr2799PZmYmKpUKj8fDpUuXcDgcmzoNKLw8kkqlQol0b2/v\n7115J5FIhA2k06dPU1lZidPppLm5mdu3b9Pb2xv1gjM4OIjH42F+fp7U1FShwfbzrgvPicnJyU9N\nLcvPz6ekpESIj8dqbwF4EtM0GAzU1tZSXl4ufL6wsIDVauXu3bsxl63wNMFgkLm5OS5cuEAgEGBw\ncBC1Wo3L5WJiYgKr1UpnZycrKyvk5uby4osvCimE3d3d676xuKlENyxG4c0Am81GMBhEJpMRHx9P\nfn4++fn5LC8vb2rRfRqv18vS0hILCwuf651KJBKhCKK8vJxXXnmFY8eOoVar6ezs5MMPP6Snpyfq\nvdwwExMTa9bOMSMjg4yMDAKBAO3t7QwPD6/J90YjGo2GsrIyqqqqyM7OJhQKEQgEBMG12Wxrkvcc\nzfh8PsbGxvi3f/s3/u3f/u2514TnRFFREfHx8UxPTzM+Pr7uxTKbSnTNZjN5eXnU1NSQk5NDQ0OD\nsLkyPz/P4OAgHR0dMdWLYWRkhK6urt9ruaNWq0lPT6epqYlDhw6xe/duxsfH+eCDD3j33XeZnp7e\n8melBQIBodw8VklPT+d//I//QXV1NUajkUAgwPDwML/+9a/5r//6r5hxSL4q4c228Kbt3NycUKW2\nnkS96KpUKsxmM6WlpVRUVFBRUUFRUREJCQkkJiYCT2JVS0tLOJ1OXC7Xpo/phgmfX5WQkPC5xQw5\nOTnU1tayb98+KisrSUhIYHx8nLfffpvLly/T29u7QVZHN+G5Eqtn5sHvPN3ExERkMhkej4cHDx7w\n8OFDxsbGoj60tFHMz88zOjoqrPz8fj9zc3PrntERlaIbPj7FaDSSnp5OWVkZBw4cYNu2beTn52Mw\nGIR458LCAlNTU3R2dtLV1RUzEyossikpKZSWlpKfn4/dbmdlZYVgMIhGo8FoNGI0GlGpVOzYsYP9\n+/ezb98+ZDKZkBb0/vvvx1yJ51dFpVJtqhaXX4T4+HhycnLQ6/XI5XIhDerhw4dYrdaYcUjWAo/H\nw8LCwoYXF0Wl6MpkMkwmEzt37uTIkSMcPXqUtLQ0ocN9mHClyS9+8QuuXbtGb29vzEyq8JJHo9FQ\nUlLCd7/7Xa5cuUJvby9ut5uioiLq6+vZuXMn2dnZwgkAc3NzWK1Wbt++zfnz52N6w+TLIJPJSExM\n3FTN3L8IDQ0NvPXWW+j1euHFHQwGGRoa2hI9p78IycnJVFRUoFarN/R3o0p0dTod8fHxFBQUcOrU\nKSorK8nJySElJUXIM3S73UxMTHD37l3u37/P48ePGRsbw+FwbJoNoi+CVColKSmJEydOUFZWJmRm\nZGVlkZKSIpwhFy4A6Ovr4/z587S0tAiescjvCOdqrlXnsmhBIpGQnJxMWVmZUBgSCASw2Wzcvn2b\n0dHRmA6pfBn0ej1JSUkbfghnxEVXLpeTlJREWVkZKSkppKWlCd2QkpOT0Wg0QgqIzWZjYGCAjo4O\n7t+/T0dHB2NjY1Gf6P9FWVlZweFwMD09jclkQqvVkpubS0pKCh6PB5/PJ5yE7Ha7sdls3Llzh0eP\nHjE+Ps79+/ex2WyRvo2oRCKRCMcXxRLh+0pLSyM1NVWYGz09Pbz99tuMj4/HpFPyVfB6vUK+9kY2\ngIqI6IZPQtBqtRiNRnbs2MG3v/1toUwvLi4OeOKVhCtLWltb+e1vf8uVK1dob2/H7XbHTPz248zN\nzdHT08OjR4+ExjdarRaNRiMcouh2u5mfn2d8fJzbt2/z7rvv0tzcHGnTox6pVEpaWhoWiyXSpqw5\nKpUKvV4vhBZWVlbo7u7m/fffj7RpUYnT6WRgYEBoHhWuVltvAY6I6MbHx1NWVsaRI0coKSkhPz+f\n3NxclErlM66+z+fDarVy+fJl3n//ffr6+pibm4tpwQVwu90MDAzw//7f/6OlpYXDhw9z7NgxYfPH\n5/Nx8+ZNrl27xp07dxgeHo6pY+bXk3DzHxERl8vFyMgINpsNi8WCQqEgJSWF2dnZdc1jjojopqSk\n0NjYyLFjx8jIyBDOQZubm2NychKHw0EwGKSzs5P29na6urro6enB5XLFtNiGCQaDLC8v09fXx8rK\nClarlatXrwo5yeFE98HBQcbGxrZESetXJVxoUVtbG3PhKHiyKpyfnxfuMzU1NdImRT2BQIC5uTmu\nXr2K0WgUeg4DtLe3r1u+f0REVyKRCPXR4SYmoVAIh8PB4OAgo6OjBINB7t27h9Vq3ZLJ3OEQQn9/\nP/39/Zw/fz7SJm1qent7uX79OhqNhtbW1pg7Fy3cV+HRo0ckJyeTk5PD0tISfX19kTYtqllaWuLK\nlSvk5ubS1NTE8ePHWVxcFA7rXA9nRrLeb32JRPLcH3i6O9TTPG3PentvoVAoIuvMTxuTaCES47IR\nYxKec+F0vC8y9zfLXAmXgod5usPcWrNZxuSzCJfNf/vb3+att96iqqqKhw8f8s477/DjH//4mcMC\nfl8+b1wilr0gLodFNpqtMOfCfRZEfj+CwSArKyucP3+e+fl5XnrpJYqKiqiqqiIxMZGpqak1j+9G\nPGVMREREJJIEAgGGhoZYWVlhaWmJiooKHA7Hl/Jyfx8iFl6IBmJhebQexGp44asgzpVPIo7J8/m8\ncVl30RURERER+R2xVQspIiIiEuWIoisiIiKygYiiKyIiIrKBiKIrIiIisoGIoisiIiKygYiiKyIi\nIrKBiKIrIiIisoGse0VaNCcyi8ndz0csjvgk4lz5JOKYPJ/PGxfR0xURERHZQETRFREREdlARNEV\nERER2UDELmObEJlMRmVlJUVFRaSkpOByuWhtbaW3t1c8/VdEZB2pqqqiqKgIgLt37zI2NvaFv2NT\nim64GbVCoUCtViOTyVhZWdkSR0xLJBK0Wi0vvPACp0+fprq6momJCX74wx8yPT2N2+2OyeNoPovw\ngYIACoUCjUZDfHy8cLzR8wifBDszM4Pf798oUyOCRCJBpVJhMBiQy+XY7fYtN0e+KjKZDLPZzAsv\nvMALL7zAysoK09PTW0N0pVIparUatVpNRkYG5eXlGAwGWlpaaGlpifkHSK1Wk5qaSkNDAwUFBchk\nMlJSUsjMzCQxMZHJyckt18Rao9GQlJSETCYjLS2Nbdu28Yd/+IfEx8d/qrgMDw9z8+ZNfvSjH2G3\n22O2wblEIkGpVFJQUMDRo0dJSEjg7/7u78QV0RckLi6Ob37zmxw5coTU1FRu3779pZ28TSG6EokE\no9FIcXExFRUVZGZmkpycTFJSEhaLBaVSSXFxMUajkd/+9rcx+wDBk4bLbrcbh8PBwsICRqMRhUKB\nQqH4TM8u1lAqlcTHx1NcXEx5eTnl5eVIpVIsFgs5OTkUFxcLx9U/D5PJhE6nY35+nsuXL9Pd3Y3X\n693gu1h/ZDIZ8fHxfOc736GhoYGJiQni4uLwer0x76CsFQUFBTQ1NXH69GkKCwuZn5+nq6uLhYWF\nL/V9US26arUas9lMWloahYWF1NfXs23bNjIyMjCbzSgUCkKhEBqNBoPBgMfj4dGjRywuLsbshPL7\n/SwuLjI2NobT6SQzMzPSJkUEk8lEY2Mj+/btY/v27RQWFiKRSNBoNGi1WoDPXEKbTCZKS0v5xje+\nwcLCAhMTE8zMzMTcslsmk6HT6di3bx+lpaV4vV7MZjPz8/Mx+4ysNbm5uRw+fJjq6mokEgkdHR3c\nv38fp9P5pb4vKkVXIpEgk8lITk5mx44dnDx5kv3795OcnCzE7qamprDb7QQCAfLy8sjOzqauro7M\nzEwGBwdZWlqK8F2sD8FgEI/Hw9zc3JZeIiYnJ/Paa6+xa9cu0tPTBbEMBoP4fD7h0NPw5+F9gKcP\nbdRqtTQ0NHD//n3u3r2L0+mMOdF9Go1Gg9lsxmw2Y7PZtvT8CWuMRCLB7/d/6r+7VColISGBvLw8\n1Go1o6OjtLa2cvv27S99dlrUia5CoSAtLY0XXniBxsZGSktLSU1NxWw2I5PJ8Pl8zMzM8JOf/IRH\njx5hNpv5m7/5G7KysjCZTJhMJhQKRaRvY92Qy+UYDAYyMzOJj4+PtDkRY3FxkdbWVkpLS0lPTwcQ\njhzv6el55tRfgMzMTHJycsjOzo6k2SJRgFQqJTc3l/r6enJycvjZz37G5OTkJ8JLMpmM6upqamtr\nSU5OZm5ujsuXL3P27Fl8Pt+X/v2oE92EhATq6up4+eWXKSsrw2KxCCLqdrsZHh7mN7/5DefOnWNk\nZITc3Fw8Hg8ymUyIbT7vaPdYQSaTodFoSExMxGg0Cp8bjUaSk5MxmUwsLi5+pUmxGZifn+fq1av4\n/X6ys7MJhUKsrKwwPDyM1Wr9hKebkpLC3r17ee2119BoNMhkMrxeLzabjdHR0Zj3csM87eFtVRQK\nBQcPHuTo0aPEx8fT3NzMwsLCJ0RXLpezZ88etm/fjkKh4OrVq1y7do2urq6vtG8UdaKbkZFBY2Mj\n9fX1GAwGAoEAq6urrK6uMjo6yvXr1/nxj3+M3W5HrVajUqm23AT6uBcHTzy5qqoqJicnsVqtXzrI\nv1lYWFjg/v379PT0oFKpgCebjCsrK89dNpvNZpRKJSdPnkSpVCKTyVhdXaWnp4f+/n6mp6e3hOjK\nZDIMBkNMrwY/j7CY7tmzB4/HQ1paGgMDA8zPzwvXhMepoaGB8vJyfD4ft27doq2tjdnZ2a/2+1/1\nBtaarKwsamtrUSqVAHg8HhwOB4ODg5w9e5YPP/wQm82Gz+fDYrGQlpYmXLsVWF1dxeFw0NzcTGZm\nprC0rqurQ6vV4nK5WFhYiHnRhSciu7Cw8MxL99OEMy8vj8LCQnQ6nZDl4ff7mZ2dZWlpactsKqnV\navLz8+nu7sbhcETanIjh8/mEf3O5XP5MrB9Ar9eTl5dHamoqer2eubm5NcsQijrR7ezs5O2338bh\ncDAzM8P4+DiTk5PY7XaGhoYYHx8Xls4qlUqI9W4lVldXuXLlCsXFxRw4cAB4MhY6nQ6VSvWJCRTL\nfNzj/zgGg4Hi4mKampqora19ZnyWlpa4cOECfX19G2XuhhMKhQgEAng8Hnw+H0qlkrS0NDQaTaRN\niwiZmZns3buXkpISZDIZo6OjjI+PP7PxLpfLycnJ4cSJE6Snp7O8vExfXx+PHj1iamrqK9sQdaI7\nPDzMysoKdrsdu93O5OQk8/PzrKysfCLpX6fTbTlPF57s0FutViYmJoRQg8jvkMvlJCUlkZGRQUFB\nAbW1tdTU1FBUVCRkv4RXUG1tbTHt8YUF1+FwsLi4iEwmQ6/XC+OwlQgXiZw+fZqcnBymp6e5efMm\nIyMjLC8vC9fl5+ezd+9ejhw5gtFopLOzk/fee4/u7u41WUFG3ch7PB5GR0cZHR195vOwa//0UtJi\nsVBQUIBKpSIYDOL3+z8z/SPWkEgkz2wYbcXYtlwuRyaTPbPaMRgM7N27l5MnT7J7926Sk5OFDdZw\nSpndbmeeaAmjAAAgAElEQVRgYAC32x3zxTSLi4sMDg5SVlaGwWCItEkRQSqVkpiYSHl5Ofv370ep\nVHLx4kXOnDnDxMQEXq9XaC1w8OBBvvGNb7B9+3bsdjvnz5/nBz/4wZrZEnWi+zyUSiV79+4lKSnp\nmQ2AwsJCMjMzUSqVOBwOuru76e7uZnFxMYLWbhzhpXX4JbNVXjbwZE6YzWZqa2spLy8nPz9f+DOt\nVkt1dTUpKSnCppFEIhEE6Ny5czQ3N9Pa2irkeovELmq1mpSUFL71rW/xwgsvoNVqaW5u5saNG/T2\n9gpZC4mJiezfv58XXniByspKfD4fV65c4fHjx2tqT9SKrlwux2g0kpmZSU1NDQcPHhTq68NLapPJ\nRHp6OoFAgLt37/Lhhx9uuUqbrSS0YfR6PQUFBRw/fpzKykry8vJISkoS/jwcXvj4EnppaYmBgQHO\nnz/P3bt3mZqa2hJNkp5GKpUKKXOxjkQiIS0tjaqqKo4cOcL+/fspLCwUUkuTk5MpKSmhp6eHQCBA\neno6Bw8epKioiEAgQEtLCxcuXKCzs3NN7Yo60VUqlRiNRnJycsjIyKC2tpZDhw5RXFyMUqkkEAhg\nMBiEpXQwGGR2dpbe3l5aW1tFr2ULoNfrKS0t5a233iIjIwOdTveJa54X6/b5fMzPzzM6OsrExMSX\nrijazCiVSlJTU2N+I00mkxEXF0dDQwMvvvgi3/zmN1EqlUJFZ0ZGBvv378dgMHDx4kX8fj+VlZXs\n2LEDs9nM8PAw77//Prdu3WJ8fHxNbYsq0ZXJZCQkJLBz506+//3vk52dLUwOj8fD4OAgy8vL1NTU\noNVqBU8mXGFSVVXFyMgIq6urW9ID3Go8HdP+tM+fngcJCQk0NDRQX1+Pw+Ggv79/w2yNFvR6Pdu3\nb8disUTalHVFq9WyY8cO3njjDY4dO4ZcLhcKaGZmZkhKSiI9PZ26ujr27duHUqkkMTGRlJQU3G43\nLS0t/NM//ROrq6trbltUiG54QyQ/P1/oE1tcXEx3dzednZ0MDQ0xMjKCy+USWhhmZWUhl8uRSCTo\n9Xp27tyJRCLB4XDQ1dWF0+mM6Q0SQOglEL5PnU5HcXEx165di7Bl68v8/Dz379/n7//+79m2bZtQ\nE//0JmrY042Li6OyspLa2lpSU1ORy+UoFIotlVYXCAQYGhpienqavLy8mL//1NRUtm/fzne+8x1q\na2uRSqUsLi5y8eJF2tracLvdHDt2jIqKChISEigtLUUmk6FUKoU9o507d/K3f/u3nDlzhu7u7jXt\n5RJx0ZVIJJjNZoqLi4Ug9vbt23E6ndy8eZOPPvqIoaEhHA4HeXl5lJaWCgPjdDqZnJxEp9NhsVho\nbGxkdnaWu3fv0tnZydjYGD6fj1AohM/ni9kmOGEMBgO1tbVkZGTQ1dUVs/FKj8fD8PAws7Oz9Pf3\nI5fLGRsbIxAIfOJFGx8fT3d3NxaLhdTU1AhZHFmCwSA2m425uTnhRR2u5lwPTy7SZGdnc+DAAWpq\nalheXubKlSsMDw9z4cIFenp6CIVC+P1+VldX2blzJyaT6ZkYt0KhIDU1lfr6es6dO7fm9kVcdDUa\nDSUlJbz55pucPHmShIQEHA4H9+/f5/LlyzQ3N+Pz+UhOTubEiRP80R/9EdnZ2aysrNDe3s7ly5eF\nKraioiL+6I/+iJ07d3Ljxg2uXLnC8vIyfr8fp9NJe3t7pG93TfH5fLjdbsFzMRgMbNu2jdLSUlpb\nWxkeHo60ietGuJqsubn5M69zOBxMTk7y0ksvbZBl0UcwGGRhYQG32y30XjCZTOj1+pgU3fT0dCoq\nKnA4HNy4cYNz585x/fp1YYNdKpXym9/8BrlcTkZGBlqtFp/Px+rqKoFAgFAoxOjoKA8ePGB0dHTN\nnbWIiq5UKmXXrl2cPn2a06dPo9VquXv3LufOnePGjRsMDAygUCgoLi7m9ddf58CBAyQlJTE1NcUv\nf/lLLl68SHt7O0ajkcLCQqqrq6mvr6egoICXXnqJo0ePCmlC169f57//9/8eU7HewcFBrl69yu7d\nu9Hr9cLnNTU1dHd3x7To/r4YDAZqamq2dEc24JlVQFh4YzWDYWJigkuXLtHV1SUUET2d0SSRSNDp\ndCQnJ5Oeno5cLufixYtcunSJyclJQqEQCwsLjI+PMzExseb2RUx0NRoNpaWlHD9+nKamJoxGI9ev\nX+fSpUvcuXMHj8fDjh07KCoqoqqqih07duDz+fjoo494/Pgx169fp7u7G6fTKZQMDw0N0dLSwq5d\nu6ipqSE7OxuJRILVaqW1tTWmBBdgdnYWq9VKfX39M58bDIaYS4IPn/OVmprK4uIic3Nzn5upkpGR\nwfbt2/n6179Obm7uBlkafYRjuiMjI8zPz2MwGCgtLaWoqCgmq/HCG+42m43FxcVnuodJpVK0Wi3F\nxcXk5uai0+lYXFzk4cOHnDt3Tqg4C68i16NbX8RE12g0cvToUQ4cOEBBQQHLy8tYrVamp6eFSrO6\nujqqq6vJy8tjbm6Ojz76iF//+tfcu3cPp9MpLI18Ph/T09NMT0/T3t7O6Ogovb29lJaWAnDjxg3u\n3LkTqVtdN6anp+nt7cXlcmE0GoVsDrPZTEJCAiqVCp/Pt+k3FLVaLRaLhczMTGpra3n8+DEtLS3P\n7SYWrioyGAzs2LGDU6dOcerUKbRaLaFQiGAwSDAYjLkX8GcRDAYZGxujv78fm81GUVERBoPhmdVR\nLDE1NfWpPRL0ej25ubk0NjZSWFiIx+PhwYMHPHz4EKvVuiH2RUx0TSYTp0+fJj8/X3hQmpqaaGxs\nRKfTkZ6eLpT+er1eLly4wJkzZ7h+/bqwOfZpdHR00N3dLezQBgKBmMzftVqtXLx4kZdffpn4+Hih\nv259fT2Tk5P86le/YmpqalPno8pkMnJzczl06BAnTpygtraWH/zgB/T29j735ONw6k9tbS2vvfYa\nR44cQafTCRVpq6ureL3emJwPn8fk5CTd3d2kp6dz8eJFrl69GmmTNpz09HROnjzJyZMnSU1Npa+v\nj3/4h39Y86qzzyJiohsMBnG73fj9fqRSKSqViszMTEKhEF6vl4GBAUZHRxkcHKSnp4f29nasVuvv\ndXhg2JuJdcJjaLPZKCgoEERXKpUK/212FAoFp06d4mtf+xplZWUYjUZqamp45ZVXWFxc/ITopqSk\nUFhYSGlpKXl5eYLgjo+P093dzb1797h58yZ2uz1CdxQ5gsGgENsNv3y2EuGOc0eOHCEhIYHR0VGu\nXr3K0NDQhmY2RUx0XS4XZ8+eZXJykszMTAwGA16vl+npaUZHR+nv72d0dJSRkRGGhoZwuVwxfxrC\nl8Hj8dDW1kZRUREZGRlAbJUGy2QyCgoKKCgoEBL6KyoqMBqNzxUNi8VCRkYGiYmJhEIhFhcXmZ6e\n5urVqzQ3N9PW1obVan2mq9RWIryqTExMJD4+npmZmUibtGE0NDRw9OhRSkpKmJ+fp7m5mfPnzzMz\nM7OhrQMiJrrT09P88Ic/pKKigrKyMjIzM3G5XLS1tfH48WOWl5c/N4wgAisrKzQ3N7N9+3Zqa2tj\n7iiWsHC6XC5MJhNqtVoQ4Y9fF17hhD252dlZhoaGePjwIT//+c9pbW3dUn05Po5CoUCj0aBWq8nO\nziY9PX1Lie6hQ4c4fvw4RqOR5uZmLly4wJUrVzbcjoiJbigUwu1209HRwcDAgNBXwe12C+32RMH9\nfHw+HxMTE4yOjuJwOEhJSYkp0fV6vZw5cwa73c7BgwdpaGh47gZQKBTC6XSyvLwsNLa5c+cODx48\nwGq1Mjs7u6UFF54UDVRXVxMKhZienv7Kx85sNtra2igvLychIYGuri5GRkYiYkdE83TDtdBb+Sjo\nr0ogEGBmZoZf//rX9Pf3C82ArFYrTqdz04dkAoEAPT09LC8vMzo6ytjYGDt27CA/Px+lUsn4+Dhj\nY2PYbDa6urpwOBy4XC6cTqfQ2MblckX6NqKCcN8BqVTKwsLCpt5g/TLcu3cPl8vFpUuXuHfvXsRE\nV7Le3qREIoladzUUCkXEJYzmMYHIjMvnjcnTFXdNTU1UVVWh0Wjo6+ujr6+PgYEB4dyv9Sh/joW5\nsm/fPg4fPkxiYiLvvPMO9+/f/0ovpFgYk/Xg88ZFFN0IEM1jAtEpuk9d94nuYh9v5r4exMJceXrs\n1iJ8Fwtjsh583rhEvPeCiMgXYb3FNZYRxy462PyJnCIiIiKbCFF0RURERDaQdY/pioiIiIj8DtHT\nFREREdlARNEVERER2UBE0RURERHZQETRFREREdlARNEVERER2UBE0RURERHZQETRFREREdlA1r0M\nOJrrpMXa8ecTzb0XIoU4Vz6JOCbPR+y9ICIi8glUKhUpKSnCKcmDg4OMjo5G2KqtgRheEBHZYkgk\nEiwWC6dOneInP/kJ//7v/86JEycibdaWQfR0RUS2EMnJyZw6dYqGhgZqa2uJi4ujt7eXubm5SJu2\nZYhq0dXpdFgsFlJTU8nNzSUYDDI3N8f8/LxwPMvIyMiWOPlXROSroFarKS0tZd++fZw8eZKioiIM\nBgOzs7NcvHiR3t7eSJu4ZYhK0ZXJZOj1eoqKiqisrKSmpoa9e/fi9/sZGRlhdHSUYDBIa2srTqeT\npaUlUXhjDIlEglKpRKPREAqFkMlkKBQKpFIparUauVzO6uoqfr//Ez1iw4dUejwePB7Pljtq/Gmk\nUikqlYqsrCxefPFFXn/9dfLy8lhcXGRiYoKOjg5++ctf0tPTE2lTN4xweCV8tNXk5CSrq6sbpiFR\nKbomk4mmpiZef/11duzYgdFoFB6+kpIS4dyvS5cu0d3dTVdXl3jOWoyhUCjIysqitrYWn8+HxWIh\nLS0NtVpNVVUViYmJ9Pb2Mjs7+4mzvgKBAAsLC3R1ddHV1cXw8PCWfSlrNBqys7N54403OHr0KNnZ\n2UilUq5du8bNmzfp6elhdHR0S72YVCoVb775JgcPHkSlUvE//+f/pLu7e8M0JCpF1+PxYLVaCQaD\nxMfHP/f0V4D6+nr+/M//nP/4j/+gra0Np9O5wZaKrBfbtm3j8OHDHDp0iGAwiFqtRq/XI5fLSUxM\nRKvVkpqaitvtJhAIPPN3w0ewO51ObDYbw8PD9Pf309/fj9VqxeVybZmTgZVKJfHx8dTX15Ofn49C\noQDA4XDQ1dVFS0sLi4uLW+ZECalUikajoby8nOrqaoLBIJWVlczMzGzYQZVRK7oDAwN0dnZSUVFB\ncXExUqkUv99PIBBApVIhkUjIyMjg2LFjDAwM4HA4RNH9FORyOUqlEqlUSlxcHImJiSQnJyOVSnG7\n3czOzmK1WllaWoq0qQLbtm3ja1/7Grt37xY+C4VC+Hw+3G43y8vLqFQqVCqV8Gcej0cIP8hkMkpL\nSwmFQszMzNDd3c21a9dYWVkRwhKxjkqlIiMjg127dlFYWIjJZGJpaYne3l7a29uxWq1MTU1F2swN\nRyKRoNFoUKvVhEIhsrOziYuL29qiGwwGWV5e5tq1a6SkpJCdnY1arWZ5eRmPx0NSUhISiQSFQoHJ\nZKK4uJjExMRImx11SKVSQWgtFgsqlYra2lqampo4evQoCoWCiYkJ7ty5w//9v/83qjZTcnNzycrK\nYnV1VfDCAoEAi4uLjI+PMzMz88z1wWAQm82GWq0mOTkZnU5Hbm4uqampZGRkkJmZiUql4vHjx9jt\ndpaXlyNxWxuKxWJhz549fO9738NiseDz+RgaGuL//J//w40bN5icnIy0iRtOONY/NTXF/Pw8ZrMZ\no9EovLw3gqgU3TCtra1kZmZy5MgRMjIy8Pl8eDyeZ5ZCHo+Ha9euMTAwEEFLo4/4+HgKCgooLy+n\ntraW8vJytFotRqORuLg44uPjkUgkBAIBioqK0Gq1kTb5Gc6cOUNbWxtJSUnMzMwI4ut2u3E6nSwu\nLj5zfSgUwuv1ChtHWq2WP/7jP+brX/86JpMJiURCQkIC9fX1dHd3b4lVUVlZGXV1dcTFxdHf389H\nH33EhQsXaG9v39IpYj6fj9u3b5OXl0dTUxOpqakYDIYN+/2oFt35+Xl6enq4evUqe/bswWw2CzuO\nYYLBIDMzM1vCc/k84uPjqaurw2w2k5mZSVlZGdnZ2eTk5JCWloZEIsFutzMyMsLjx48ZGxtjamoK\nh8PB9PR0pM1/hp6eHmw2G3q9HpfLhc/nIxQKEQgEcLvdwmbq81CpVOTl5REMBlEoFMLLZWlpCZvN\nFvObRvHx8Rw8eJBDhw5RXV3N4uIily5d4oMPPuDhw4db/lkJBoP09vYyPj6OWq2mvLyczMxM1Gr1\nJzZl14OoFl2AsbEx3nvvPXw+H7t376a4uPgZ0ZXJZCQnJ2MwGLbs21smk2E0GqmuruY73/kOmZmZ\npKSkkJyczPz8PG63G6vVisfjoa2tjdbWVkZHR2lpacFms0VlfHNpaelLxZgNBgOZmZns37+fkpIS\nYRN2YmKClpYWbt68yfz8/FqbGzWoVCry8/P53ve+J8S0e3t7+eijj2hvb9/yggtPRNfhcLCwsIBK\npaKkpISCggISExMZHx9f903FqBfd6elpfvvb39Ld3Y3D4eBb3/oWmZmZyGQy4ElKzKlTp5iYmNiS\nteNSqRSdTsfhw4d5+eWXOXbsmBD77Orq4oMPPqC9vZ2pqSm8Xi8Oh4O5uTkCgQBer/cTO/+bGblc\nzvbt23nttdc4ePAgaWlpwp/99Kc/5cyZM1it1pi654+Tnp5OfX09paWlxMfH09/fz+3bt5mdnY3p\n+/6ySCQS5HI5ubm5lJWVMTk5ue5OSNSLrtlspqSkhIyMDIqKioRYZJhAILClwwt6vZ4jR45w6tQp\nysrKePDgATdv3qS3t5eFhQVGRkZwOp1CapXX6/3MpflmxGg0kpmZSXl5OQcPHuTIkSOkpKSgUCiw\n2WxcvXqVGzduMDo6GpVe/VqhUqlIT0+nqKgIjUaDVCrF6/UyOzvL7Owsbrf7mevT0tKorKzEYDAg\nk8lwOBxcu3Zty+U0SyQSVCoVarX6GW1ZL6JSdKVSKVqtluLiYmpqaqitrSU5OVmYIE/j9/vp6enB\nbrdHyNqNJfxmDocQUlNTyc7OJhQK0dPTw5UrV7h27RqDg4MxJ67PQ6/XU15ezokTJygvL6eiokLo\nnBUIBJidneXq1asMDg7GfAGNXC4nKSmJnJwc5PInj/bKygo2m02Ii6vVajIyMpBKpZSVlfG1r32N\npKQkFAoFDoeD1NRU7ty5w9jYWMzPn6fDCBKJBKl0Y/p/RaXoKpVK0tLS+O53v8upU6dISkpCJpM9\n9y3k8/no7+//RApRrKJQKDCbzRw7doxDhw6RnJzMj3/8Y86cOcPc3Bytra14PJ4t462kpaVx9OhR\n/tt/+29IpdJn5kh4483j8SCTydBoNHi9Xvx+P8FgMOYKAsLlrVlZWYLoLiws0Nvbi9vtFjI4jh8/\njlarpbS0lIMHD5KQkCAUTXznO9/he9/7Hu+++y7T09MxN0bRQNSKbnp6OmlpacTFxQnx2+ehVqs5\nePAgU1NTNDc3b6CVkSE7O5sTJ07wyiuvoFaraWlp4fHjx0xMTLC6urqlBBegpKSEkpKS53opMpmM\ngoIC/uZv/oa+vj66u7vp7Oyko6ODkZERZmdnI2Dx+iCVSsnPz6eiooL8/HxkMhkul0tIrwsEApjN\nZgoKCqioqKC6upqsrCxMJhPBYBCn00kwGCQpKYlXX30VgHfeeUeMBa8DUSm6Xq+X8fFxHj16RFpa\nGtXV1Z8qvHK5nOzsbMxm8wZbGRmys7N59dVXKS0tpaOjg4cPHzI5OYnT6dySXonL5WJiYoLBwUHi\n4+PR6XQolUrgieen1+uF4pnCwkK2bdvG2NgYbW1tPHz4kO7ubhYWFjZ1rDdchRfOWAkXEvX19dHV\n1YXT6USr1VJRUcGBAweor68nLS2NQCBAZ2cn7e3tLCwskJqayte//nWys7PJyspCqVRuSIwzUkTq\n3qJSdD0eD/39/Zw7d07oNqVSqQRvRiKRkJiYiF6vRyKRoNVqhQct1omPj6eqqgq1Wo3X68XtdqNS\nqVAoFDGff/o8rFYr169fRyaTkZeXR2JiIjqdDrlcLhSDGI1GzGYz8fHxFBcXC/HvwsJCzp49S2tr\nKxMTE5G+lS+NUqkkISGB0tJSUlJSWF1dZXh4mBs3bnDv3j3cbjeVlZU0NTVx5MgRSkpKmJmZoaOj\ngxs3bnDr1i20Wi379u3D7/fj9XqF/7bii3y9iUrRDfPo0SNhcoQfJHiybHzzzTfZvn17TL+Jn4fL\n5WJgYIC8vDxqampQKBS43W5u377N+Ph4pM3bcGw2G1NTU1y5coWkpCShrNNkMlFVVUVjYyNNTU1C\nzDI8X4qKikhJSaG0tJR/+qd/4r333tu0AmM0GqmqquL111+nqqoKp9PJ5cuX+cUvfkF3dzdJSUl8\n4xvf4OjRoxQVFeHz+ejo6OBXv/oVv/zlL0lISOCNN97gpZdewuv1MjAwQF9fHzMzM5t2TH4fInVv\nUS26fr+fyclJbt26hVwuFx4YqVT6TC+B1NRUUlNTMZlMLCwsRNjqtcVisaBWq3E4HPj9fjo7O/n7\nv/979u3bR0NDAyUlJfzZn/0ZarWas2fPMjs7G9MPyscJBoPPeGYOhwOFQoFSqaSnp4eWlha6u7up\nq6ujsLCQ5ORkIQMkLFYHDhzAbrdz7969TRm/DPeZMJlMKBQKZmdnee+99+jr6yMvL49vfetbNDU1\nkZOTg8Ph4D//8z+5c+cOo6Oj5OTk8N3vfpddu3ah1Wq5fv06Z86c4ebNm1tqHs3Pz+NwODZkPySq\nRRfA7XZ/Ir9QJpPh9XoFITYYDBgMhg1tWrGeyOVywVPLz88nFAoJnuzk5CTnz59ndnYWg8FAdXU1\njY2N3Lt3j5s3bzI3N7elHpanWV1dZXV1Vfh/u92OzWbDbrfT09PD3r17OXz4MHFxcSiVSuRyORaL\nheLiYkpKSnj48OGmFF2VSkViYiJKpRKXy8Xw8DCdnZ34fD4KCws5cuQIOTk5zM3NcevWLd59911s\nNhtZWVmcOnWKF154gWAwyIMHD3jvvfe4fv36lls1zc3N4XA4NuTZiXrR/ThyuRyDwcCuXbvYtm0b\nPp+PlZUVPB7PpnxgnofJZKKmpoa//uu/Jjc3l7GxMdxuN0tLSywuLuJyuWhtbWXv3r1CfmHYuxN5\nFpfLxb1793jw4AGDg4Okp6dTVVX1zFjpdDqhKc5mRCaTodVqkUql2O12ent7hcbv6enpWCwW5HI5\nnZ2dXLx4EbvdTkJCAgcOHOCtt95CoVBw+fJlzpw5w7lz52LmOfoiLC0tbZjDsulENzExkZdffpmc\nnBzgiXdz+/ZtOjo6Yia0cOjQIf7iL/6C3NxcHj58yAcffMDly5eF3hJyuZw9e/ZQUVGBx+Ohr69P\nKBDZql7u5xEMBllYWMBqtVJYWPhMtsvCwgIzMzMxk2r3vOOLQqEQZrOZffv28eqrr6LX61Gr1YyN\njfHv//7vtLW1YbPZtqTgbjQRF12lUklWVhYNDQ0MDg4yNDT0qdVlSqWSjIwMTpw4QXp6OvAkvezR\no0cMDw9v6rSfp/F4PCwuLiKXy8nJyeHw4cPk5OQIZbwej4fGxkaKi4uZmJjg3LlztLW1sbS0tCVE\nV6VSYbFYKCwsZHl5mampKcbGxj7z74T7LxuNRmFDNixG09PTG9LoZL1YXFykp6eHlZUVIVySnZ2N\n0WgUwg5SqZTMzEyMRiMpKSkEAgF6enq4ePEiN27cYGJiIuYr0D4LqVT6mfUAa0nERTctLY2mpib+\n8A//kDt37vDRRx+xvLwspK5IJBLUajUqlYrU1FR27NhBfX09ZrOZ5eVlxsbG6OrqirrWhF+FcLqc\nXq8nIyODAwcOcOrUKQDhpAedTofD4aC5uZkPP/yQ/v7+mHnpfBbh5iR1dXXs3buX+/fvc/fu3U8V\n3fCmmcViIT8/n4KCAjQaDfDE+3W73YyPjwuHnW5GFhcX6e/vZ3l5mYKCAiorKzl69CharZby8nLU\najVSqZTk5GSSkpIIhUJMTk7S1dXF+fPnmZyc/MS+yVZDr9cTFxe3IVV4ERfdpqYm/uAP/oCdO3eS\nnp6ORCIRKmnsdjsKhYK8vDzS0tI4cOAAR44cwWw2I5PJGBkZ4YMPPmBkZCSmJk1vby9OpxOv10t5\neblwKvLTndX6+vo4d+4c7733nhDzjXVkMhkmk4mXX36Zt956i9zcXO7du8fQ0NCn/p3wpuSRI0d4\n6aWXqKysFPK9w9kx/f39DA0NbVpPNxQK4ff7WV1dJRAIkJGRwf/+3/8beOLBhT17+N3JCR0dHdy6\ndYvHjx9v2pfNWpKZmUlxcbFwNuN6EnHR7e7uprm5mczMTCwWC6dPn6ampgaHw4HNZkOhUFBcXIzJ\nZCIlJYWkpCSkUimzs7M8ePCAd955B4fDEVOxqGAwyNzcHOfPn+f27dsYDIZnNnpCoRBLS0tMTk5i\ns9m2TOmvyWTilVde4cCBA2RlZaFSqYSc3HBPhbBwKhQKKisraWxsZPfu3eTl5T3TEtTv9+NwOPj5\nz39OW1vbphVcQKg+++lPf8rY2Bh1dXVkZ2cL9+rz+WhraxOa1ttsNh49ekRnZ+eWmDcfJxQKsby8\nzPT0NJOTkyQkJAjhhS3RZcxqtfLgwQMqKysxGo3k5eVx4MABXC4Xs7OzyOVyUlJShLhUOC/z8uXL\nvP/++3R1dQmnCsQSPt//b++9gtvMzvv/D3ohAZAgQYK99yJWUSLVV8XSStoia+3Ym8TxTOJMiuNx\nrpzc5MI3uXA8uUpmfnacSVx2vVmvJHu1VKNESlqJYm+ixE5RLGABQRAEQKL9L/Tn69VK20WABN/P\nzM7sUljwvEfn/Z7nPOcpHsbGxkI9jE2FTCbDaDQSHR1NREQEAHv27EEqlTIxMSFEsKynxVZVVVFb\nW/uxOggAACAASURBVEtZWRkajQaZTCYUwRkeHqaxsZH6+vpPtZS3Ah6Ph/n5ea5evcrMzAw9PT0k\nJSU9tcH09/czNTXF/Py8YNBspkakwWb93mRpaQmj0YjBYCA2NnZ7iO78/DwPHz6kpaWF2dlZ9u3b\nx8mTJwUf1HoKMCD0yJqamuK///u/+eCDD0I8epFgsrq6yv3796mpqaGoqAiVSsXrr7/OoUOHBBfL\nuuhqNBoSExOJjY0F/thJ2Ol0YrfbuXr1Kr/4xS+4f/9+UFq0bDSBQICRkRFGRka4cOFCqIez6fH7\n/U8ViDKbzaSnpwelvGPIRRdgdHSUn//85wQCAWZnZ4W20ImJiRQUFFBWVgY8iVQYGBjgJz/5Cd3d\n3aEcskgIcLvd3L9/nzt37hAfH09lZaUQt52dnf3UUXk9dnkdv9/P2NgYjY2NtLe309raSn9//1PJ\nFCLbC5vNxsTEBJmZmSgUiqAlV20K0V1dXWVmZgapVEpra6tQfESv1xMXFye0XfH5fFgsFpqamrZN\n/VyRP7J+8XXlyhUWFhZ48OABSUlJKBQKVlZWSE1NJSIiAq/Xy9LSktDEcmpqisXFRcGVNTU1xdzc\n3La4fBT5ZEZGRrhy5QpJSUlMT08zMjISFB+3ZKN9oRKJZNM6WwOBQEhSkDbznEBo5uWLzIlUKsVo\nNFJQUEBWVhYqlYqlpSXy8/PR6/WCj9Pr9bK8vMzIyAhzc3NC5+Mvg7hWnmWrz4lWqyU9PZ033niD\nubk52traaGlp+cqX8p81L6LohoDNPCew+UX3I//PUxEd6//9SRlZXwVxrTxLuMzJuh/3RayT//97\nPnVeNoV7QUTky/DxlyTcIlhEgkOww+aC04lNRERERAQQRVdEREQkqGy4T1dERERE5I+Ilq6IiIhI\nEBFFV0RERCSIiKIrIiIiEkRE0RUREREJIqLoioiIiAQRUXRFREREgogouiIiIiJBZMPTgMXc8WfZ\nzHMCW6f2QjAR18qziHPyfD5rXkRLV0RERCSIiKIrIiIiEkTEKmMiItsQvV5Pfn4+dXV15Obm0tTU\nxOXLl1lYWAj10MIeUXRFRLYRcrkcvV5PXV0ddXV1VFdXYzQaGR4eRqFQPLcesciLRRTdLYZEIkEm\nkyGVSoV/PtrBdL1bcji1pN8I5HK5IDIej+ep9u3hikQiQafTUVJSwl/91V9RVlaGxWLhzp079Pb2\n4nA4wn4ONgNi54gQ8FXmxGw2U1ZWRnZ2NikpKZjNZkwmE3K5nJWVFSYmJvjNb35De3v7l266GO7R\nC0qlkpdeeonXXnsNvV7P+fPnqa+vx2azfaLobMW18nGSk5Opq6vje9/7Hg6Hg/b2dq5duya0ZbfZ\nbF9osw6HOdkIxM4RYYBUKkWpVFJaWsrOnTvZvXs38fHxqNVq5HI5BoOBmJgYlEolCwsL+P1+NBoN\nDQ0NoR56SFEqlahUKqHLq9/vx+fzYTabqamp4dixY/j9fu7evRvikW4sEomEiIgIampqOH36NGaz\nmbfffps//OEPdHd3bwsrfzOx5URXJpMRGRkpHK0VCgVarRapVEogEMDj8YRdp1eNRkNGRgavv/46\nhw4dIj09ndnZWcbHx3n8+DEAGRkZ5OTkkJKSwhtvvIFMJqOjo4Pl5WW8Xm+In2BjWV8HERERKJVK\noedVVFQURqMRg8GARCJhZWWFxcVFioqKKCkpIS4ujkePHrGyssLq6mrYCo9cLiczM5NDhw5RU1PD\n3bt3qa+vp729PWyf+fOw7qrTaDRERESg0WiQyWTAkw7ly8vLOByOF/7+bDnRNRqNHD16FI1Gg8Fg\nICUlhZqaGgwGAy6Xi5mZGX784x9z586dUA/1hZGRkcE///M/U1hYiMvl4u233+bSpUsMDQ2xvLwM\nPHE77N+/nx/96EcYDAbKy8s5c+YMFy9eFFrahysRERGkpKRw8OBBsrKy0Ov1AMTGxmI2m4mNjUUi\nkTA4OMj169dJT08nLS0Nv9+PzWZjbW1NeNnCDalUik6n45vf/Ca5ubm0tbXxk5/8hKGhoW0tuAAK\nhYLY2Fiqqqo4fPgwVVVVwgb94MEDLly4wKVLl5ienn6hv3fTi25kZCRms5no6Gji4uLIy8vjyJEj\nqNVq1Go1UVFRpKSkoFar8Xg8JCcn853vfIfIyEiuXLkS6uF/ZWJjYyksLGTnzp04HA7a2tr4zW9+\nw+DgIIuLi8IuvLi4iFqtpqmpiZ07d5KamsqRI0e4e/du2IuuwWCgpKSEI0eOkJubi1qtBkClUuH3\n+7FYLMTFxZGfn4/BYECn0xEXF4fL5aKjo4NHjx59af/3ZichIYF9+/ZRW1vLxMQE586dY3BwEKfT\nGeqhhQyJREJcXBylpaXU1tZSVlZGQUEBSUlJqNVqJBIJ0dHRREZGEhUVxfXr1+nq6nphv39Tia5c\nLsdkMhEfH49SqUQikWA0GsnOziYxMZG0tDSysrLIysoiIiIChULxzP8fHR3NiRMnmJmZCQvRNRqN\nJCcno9fruX//Pnfv3uXDDz985nNOp5OxsTGuXLlCamoq+fn5FBUVodPpwjoMKDY2lqKiIvbt20d5\neTnR0dHY7XZ8Ph8zMzOMjY3R0dFBTk4OJSUllJeXC1bto0ePuHv3LiMjI6ytrYX4SV48EomE1NRU\nTp06hdls5tq1a1y7do2VlZWwXQ+fhUQiQa1WU15ezssvv8zRo0dJSEhgbW2Nubk57HY7BoMBg8HA\n3r17USqVLC4u0t3d/cLmbNOIrkQiQa/Xc+LECb75zW8SHx+PTCZDqVSi0+nQaDQoFArcbjfj4+Mk\nJycTHR393O9RKpXPCPJWRSqV4vF4sFgsdHR08PDhw0/87NLSEnfv3uWVV14RLt80Gg1KpTJsLbna\n2lq+/vWvc/LkSSIiIhgcHKS1tRW73c6NGzdobm7GarWSlpbGq6++yj/+4z+i0+nwer3Mzc3R2toa\nticBmUxGYmIiu3fvZmJigrGxMZaWlrat4MITl0JMTAxnz57l0KFDxMfHs7KyQm9vLy0tLbS2trJz\n50727NlDeXk5KSkpgha9KN/uphBdvV5PZmYmp0+f5sCBAxQVFQlmvlQqRS6XI5PJuHr1Ku+//z4W\ni4WYmBiio6OJjY3l+PHjpKSkoNFoALBardhsthA/1YvBYrFw/fp1Hj9+zMjICBMTE5/4Wb/fj8vl\nwufzoVQqMRqNlJWVMTk5+alivZXRarXo9XoiIiJwOBy0trbyy1/+EpvNxuTkJHa7HZ1OR0VFBeXl\n5UREROD3++nv76e+vp75+fmwvWgsKytj586dKBQKrl27Rk9Pz7YWXICCggLefPNNdu7cCcDt27d5\n++23GRkZYX5+noWFBWZmZvD7/eTn56PX6zGbzcTHx2OxWF7IWtkUomsymTh06BCnTp0iLy8PrVYL\nPBERq9XK2NgYdrudc+fO8d577+FyuVCr1aSmprJr1y7BivP5fLjdblpaWujr6wvlI70wlpaWcDgc\nDAwM4PV68fv9n/hZr9eL3W5naWmJ1dVVtFot2dnZmM3msBXdxcVF5ubmWFlZEWJMV1dXefjwIaur\nq8TFxVFdXc2RI0fYsWMHgUCA4eFhbty4weXLl8Pa8issLCQ3N5eFhQWam5sZGxsL9ZBCjslkora2\nlsTERB49esSdO3e4cOECs7Ozwme8Xi/FxcV4PB7hxLju7nwRhFx0JRIJycnJnDp1ioyMDLRaLYFA\ngEAggMvloq+vj1//+tc8fPiQoaEhYXKWl5fJzs6murqaxMRENBqNEL3w3nvvcePGjdA+2AvC7/d/\nqtB+lPVwuampKaxWKyaTCaPRKNzmhyODg4N0dXVRU1NDeno6VVVVLCwsMDU1hdPppLi4mD/90z9l\n586dREVFMTs7ywcffMD58+ef6xsPF6RSKenp6RgMBnp6ehgeHn7q9CeRSARff7huOs9jPWMzEAhg\nsVjo7e1ldXVVOFXLZDJUKhVyuRy/34/D4cBms+FwOD73e/hZhFx0U1NTKS0tJScnR7BwvV4vNpuN\nc+fO8cEHH9Da2orT6XzmxlWlUmE0GoWLEbvdzr1795icnMTtdgf9WUJNIBDA5/Ph8/le2ALZ7MzO\nzjI0NMTg4CBxcXGYTCZKS0spKysjPz+fgwcPUlRUhMFgYHR0lAsXLnD+/Hn6+/tDPfQNQ61Wk56e\nTn5+Pk6nk//93/9lbm7uqc+kpqai1WpxOBxYLJawvEh8HuubDTwxUlwuFzKZjJiYGFJTU6moqCAz\nM5P8/HwsFgtvvfUWly9f/sLZep9GyEW3sLCQqqoqjEYjcrmc5eVlHj9+TGtrK++//z7Nzc1PLZj1\nizKTyURubi5paWmoVCrsdjujo6O0trYyPz+/bWsPrO/Y6wkCXq83bH2W8CRqY3FxkYWFBTweDzEx\nMeTl5XH27FkyMjLIy8sjMjKSx48f09LSQkNDAwMDAywtLYV66BtGZGSk8E5NT0/T2dnJysoKJpOJ\nrKwsSkpKSE9PR6vVYrVaaWpqor+//6kjdriybqgpFApMJhPV1dXEx8cTHR1Neno6OTk5PH78mP7+\nfhobG7lx4wYjIyN4PJ4XNoaQi25qaioZGRkEAgFWV1cZGxvj1q1bvPfee/T09DyzQyuVSuLj46mp\nqaGuro6MjAykUimjo6M0NzfT1tYWNpdoXxSpVIpGoxEiFgKBAMvLy09l561n4SgUCpRKJX6/H7fb\nvWVTQf1+P6urqzgcDnw+HyqViqSkJM6ePSv8udPppK2tjcbGRrq7u3E4HCEe9cai0+moq6vD7/cz\nOjqK1WrF6/WSk5PD6dOnOX78OGq1GplMxtraGrGxsZw7d064CwhnVCoVsbGxyOVy0tLSOHXqFBKJ\nBLPZTExMDIFAgJ/+9Ke8++67tLe3b8gYQi66fX193L59m+joaFwuF++//z6//e1vefz48XMDuKOi\nojh8+DDf+MY3qKioQC6XMzMzw4ULF3jrrbd4/PjxtnQtAMIJIDExEaPRKFworacKw5Ojp9FoJCkp\niczMTFwuF52dnVgsli07b263m/n5+eceke12O729vVy6dIlbt26xsLAQ9qcgnU7H/v37aWxs5Pbt\n23i9XmQyGTt27KC0tJRf/OIX9PX1IZfLKS8v57XXXhPmsLe3d0tuvp8Xj8fD8vIyarUavV6PTqcD\nEHy46/G6G1lXOOSiOzAwgMfjoa+vD4/Hw8DAAKOjo88VAJPJRFVVFWfOnKGkpISoqChWVlZobW2l\nvb2d8fHxLSscX4X1/PHY2Fiys7OJjo5GLpfj8/moqalBKpUyMzNDTEwMer0eo9FIYmIier0ei8VC\nbGwsFy9efOHpjsFAIpEQGRkpZBN9lNnZWSGErLOzk6mpqbB2tcCTlOj4+Hhh07Xb7UIGllwuZ3x8\nnIaGBh49eoRUKmV+fp6CggISExOpra2lv78/rOfo/v37/Ou//ivp6elCmvjBgwdRq9XY7Xbhwj6s\nRXd2dpbZ2Vmam5s/87N5eXkcP36curo6tFotS0tLDA4O0tDQwIMHD8JScD8aq6xWq9FoNGi1WiIi\nIpDL5UKxl3UhzcjIIC4uDngSCL5nzx7S0tKYn58nKSkJQLB8nE4nNpsNg8GAXB7ypfClSEpKori4\nmJKSEiIjI5/6s4WFBbq7uzl//nxYro3nERkZKRyfV1dXcbvdSCQSNBoNk5OTLC4u8uDBA+FUsLq6\nyvXr1zly5AhVVVX84he/CPETbCwjIyOMjY2RkpJCQkICJSUlVFdXo9PpWFxc5MaNGwwPD2+oC2pL\nvWn79u3jL/7iL1AqlcATK/m//uu/hISJcGTdijUYDKSnp5OVlUVOTg7FxcUYDAZBdBMSEoiJiRGy\n+OCPOeZqtRqTyYRUKqWjo4MPP/yQzs5ObDYbCwsLW/r2+tSpU3zjG99g586dzxSt2W7hUIBwbJZI\nJFitVqxWq1B/oqmpCeCpS6HV1VVu3brF7t27SU9Pf2GxqJsZv9/PxMQELpcLk8kkWPazs7O89957\nPHr0aEN//5YQXaVSyb59+ygrK0OtVgsvktVqpaenB7vdHlYhUlKplKioKGpraykoKCA9PR2z2YxO\np0On06HX64mOjn6q64HBYECr1QohMRaLhe7ubrq6uhgaGhI2pYWFBWZnZ1lYWGBtbY21tbUtd3mi\n0WiIj4+nuLiYI0eOUFhYiEwmw2KxMDc3x+rqqpDVGBERsS2EZJ2PhkQ5HA6Wl5eFmPf1v+ePbkR+\nv5+FhQWcTqcQ8bId8Pv9givmo22K1tbWNlxLtozoVlZWkpGRIfxsdHSUnp4exsbGtpxofBbr5SpP\nnjxJYWEhcXFxSCQSwUc3PT0tBP/b7XaWl5cpKyujuLiYiIgI3G43AwMDvPPOO7S3tzM6Osri4mKo\nH+uFYTQa2bFjB2fOnKGiogKZTEZnZycdHR0sLCwQHR1NVlYWXq8Xj8ez7axdeCK+65sqIMRvP+9z\nWq0WmUwW9heMH2e9BrNEIsHhcLC4uIjP59vw9bLpRVcmk6HVaklNTSUqKkqo/n/t2jUuXbrE7Oxs\n2L1UO3fu5Ac/+AElJSVIJBJmZmbo6urCYrEwMDBAR0cHHo+HpaUl5ufnsdvt/PCHPyQ5ORmNRsP8\n/DwtLS38+te/xu12h90pICUlhbq6Os6cOSMUa//Nb35DfX09er2eo0eP4vF4sNlsYbk+PosvYtnL\n5XJycnIwGAzbqtzjRzPy/H4/k5OTDA0NBeV92dSiK5fLSU5OZteuXVRVVWE2m7Hb7XR2dnLjxg0e\nPHgQli9UXFwcubm5OBwOLl68SH19PQMDA7jdbpxOp3Bk9Pl8REZG8sorr1BdXY1Wq8Vut/Puu+8K\nl0fhJLgqlYqMjAxOnTrFyZMnUSgUvPvuu/zhD3+gubmZ1dVV9u7dy8svv4xOpxNSOMNxjXwan/d5\n163cPXv2ANDR0RFW6+WTkEgkmEwmduzYwa5du/D7/XR1dXHz5k2mp6c3/NJ1U4quRCJBLpezY8cO\n6urq2L9/PxkZGVgsFlpaWrh8+TKtra0bGtYRStRqtZBF1d3dzY0bN5ifn3/qZZJKpeTk5FBXV8fL\nL79MUVERdrudmzdvcunSJXp7e8PuBVKpVBQUFFBSUkJiYiIzMzM0NTXR1NTE/Pw8hYWF5OXlkZ6e\njlKpfGqD2i64XC4WFxdxOBzExcWRmJjIyMjIM5+TSCQkJSVRU1NDaWkpo6Oj3Lp1K+xdDFKplIiI\nCI4dO8aRI0fIz8+nv7+fpqYm2tvbcTqd29PSlUqlaLVaDh8+zNmzZyktLcXn83H9+nX+53/+h4aG\nhrDz434Un8+Hx+MRit0oFArMZrOQ0uv3+4mKiuKll17izTffJC8vj9XVVe7du8fPf/5z+vr6wsqH\nu45SqSQ9PR2TyYTP52N0dJSHDx9isViIioqioqKC3NxclEolKysrzM3NPbNZhTvrafTj4+NkZGRQ\nVVXF/Py84DpYbz1vMBjYtWsXr732GhqNhr6+Ppqbm8Nuo/44Wq2W9PR0vvnNb1JVVYXNZqOhoYGm\npqbnbk4bwaYU3fUOEMXFxeTm5uL3+5mfn6etrY1bt25t2fCmz4vD4cBqtZKQkMDLL79MSkoKERER\nTE9PMz09zfLyMseOHaOyspKkpCSmp6f5/e9/z/vvv09nZ2dYNeX8OB8PA5NKpUL5xm9/+9sUFBTg\ndDrp6uqivr6eu3fvhnWw/8dxuVyMj49z7tw5jh8/zre+9S18Pp8gqPHx8aSkpHDs2DEKCgrw+Xz8\nx3/8B83NzdvCp5uZmcmbb75Jbm4uVquVxsZGLly4wOjoaNDGsOlEV6vVkpyczO7du8nIyEAikTA1\nNcX//d//cfPmTaERYzhz9+5d/u3f/o19+/ZhNps5duwYSqWS5eVloZaCSqVieHiY69ev09vbK4SG\nhXMrlnUfpFKpxOfzsbKywoEDBzhw4ADl5eUUFhYyMTHBlStXaG5upr29nZWVlVAPO6gEAgEWFha4\ndOkSWq2W8vJyvvOd73DkyBHgScaaVqvF7/fT2dlJS0sL9+7dY3Z2NuytXLPZTEVFBS+99BLR0dHC\n88/MzAT15LzpRDc+Pp6dO3dy4sQJkpOTWVpaorOzk9/97nd0d3eHenhBobe3l4mJCSYnJ9m/fz95\neXlP3UjL5XJ6e3uFMKn1gt3bhUAgILigjhw5gslkIiEhgYWFBe7cucM777xDX1/ftslC+zgul4ve\n3l5UKhVut5vDhw+Tk5ODRCIRCgQ1Nzdz48YNPvzww7A/CawnEFVUVLB3715ycnKYmJigvb2djo6O\noK+TTSe6hYWFnDlzhgMHDhAREUFnZyf19fWMjo5uK6vFbrdz4cIFLl68+Nz24D6fD6/X+4nxl+GI\n1+tlenoah8Mh+CQlEolQf/nq1atcvXqV/v7+bbUJfRKdnZ309fXxn//5n8/82XoMc7gLLiCUcTx7\n9iwnTpxgbW2Nd955h9///vd0dXUF/f3ZNKIrkUiIjY0lPj4evV6PUqnkwYMH3Lp1i56enm0luICQ\nHRPu/usvgtPp5N69e0RHR+PxeNi1axdWq5Xx8XH6+vqEl8jtdoeti+WLsL4hb1eLH57E+ZvNZo4e\nPUpubi4ej4euri5u377N0NBQSAyWTSO6UqmU7Oxs8vLyiIuLY3V1lba2Nm7evMnw8PC2XjgiT1hb\nW2NkZIRr166xuLjI2NgYVquVR48eMTg4SFtbG1arNdTDFNlExMbGsmPHDk6ePInRaOThw4ecO3eO\n+/fvh6yQ/aYS3eLiYqqrq8nIyGBycpI7d+5w586dsC1mI/LFWK8h0NXVRVdXV9hXxBL5akilUvLy\n8njppZfYv38/Y2NjNDQ08LOf/eyFdoL4omwa0fX7/QwODjI9Pc38/Dy/+tWvuHfvnmi5iIiIfGGU\nSiXJyckcOHCAgoICLly4wM2bN2lubg55PY5NJbpDQ0OcP3+enp4erl69ytjYmOjTFBER+cL4/X6h\nm7jdbmdgYICHDx8yPT0dcn+/ZKMHIJFINu2NRiAQCEnNv808JxCaeRHn5Pls5nkR5+T5fNa8bLjo\nioiIiIj8ke1TtVhERERkEyCKroiIiEgQEUVXREREJIiIoisiIiISRETRFREREQkiouiKiIiIBJEN\nT47YzDF1Ypzh8xHjdJ9FXCvPIs7J8/mseREtXREREZEgIoquiIiISBDZNLUXREREREJFXV0du3fv\nJjExkStXrtDW1sbs7OyG/K4tLboKhQKdTkdubi5zc3NMTEyIBXIAvV5PWloaRUVFADx69IgHDx6w\ntLS0bbpMiIh8EfLy8jh58iR5eXlERkYSCAS4dOnShhTHCZl74aM9v74sWq2WnJwc/u7v/o5jx44R\nFRX1Qr53qyKRSJDL5WRkZPCtb32LX/3qV/z617/mBz/4AQUFBURERCCVih4lkaeRy+Wo1Wqh6ed2\nWyMSiQSdTkdcXBwmk4nTp0/zta99DZlMtiF6EnRLV6VSoVQqkclkuN1u1tbWvnQX0tjYWCorKykp\nKSEQCDA+Ps6VK1e2rbUbHR0tdDvdv38/fr8fqVRKSUkJ3/nOd7h16xZ37txhaGgo7Du/inw+1Go1\nBw8e5MiRIyQkJNDY2Eh9fT1jY2OhHlpQkMlkGAwGUlJSSEpKQiqVEh0dTXR0NDKZbENOhkEVXYlE\nQkFBAcXFxcTFxdHW1sb9+/eZm5v7Ut/n9/vx+/1ERkZSWVmJxWKhtbUVq9Ua0srwoSA+Pp6ioiIO\nHDhAXV0d2dnZwJNuCwkJCezbt4+EhARcLheTk5PbouecSqUiJiaGrKwsMjIyiI6OJhAIMDk5SV9f\nHw8ePAj1EEOKVCpFp9Nx8OBBvva1rxEXF0dsbCw2m421tTWmpqZCPcQNRyqVEhkZicFgICIiAgCr\n1YrVat0wwyTooltWVsaf/MmfUFpayttvv43H48Fut3+p7q0ulwuLxYLH4yE3NxebzUZMTAwOhyPs\nRXfdlaBSqYiKiqKyspLa2lp2795Nbm4uer2etbU1JBIJarWazMxMMjMzaW1tpbGxEafTGfJizhuF\nVCpFo9GQkpJCcXExBw4coLq6mpSUFKRSKQ8ePOCXv/wlQ0ND26Ib7ieh0WhITExk165dpKSkoFKp\n2L17N52dnYyMjGwL0ZXL5cTFxaHT6YSu24ODgwwMDOD1ejfkHQnZRZrBYODVV1/FarWyvLxMT0/P\nF/4OpVKJwWB4bovycEej0WAymSgqKuLP//zPycnJISYmBoPBgEqlwul0Mjc3h0QiwWAwYDQaQz3k\noKHVaqmtreX111+nuroap9NJc3Mz9fX1mEwmDh06RGlpKWazmZmZmW0pvFKplJSUFA4fPkxSUhJq\ntRqPx8PS0hKjo6PMzMyEeogbjkQiISoqiuPHj5Obmyv8vL+/n/v372+YURIy0VUoFMTHx3Ps2DE8\nHg9Wq5W5ubnP7Y+Vy+WkpKSwZ88edDodABEREZSUlGCz2cL2+GwwGEhPT6eoqIiioiKKi4uprKzE\nYDCgUCiQSqVIJBJmZma4du0aEomEHTt2sGvXrlAPfcNZF5KqqipeeeUVjEYjvb293Llzh/7+fjwe\nD1lZWezduxeVShW2lv7nITY2lp07d/L6668TGxuLVCplbm6Oc+fO0dHRwcLCQqiHuOFkZ2dz8OBB\njh49SmpqqvBzl8uFy+XasN8bdNFdW1vD6/UikUhQKBSUlZWxurpKf38/d+7cYW5u7nP5UjQaDZmZ\nmdTV1REZGQk8sXByc3Pp6OjY6McICXq9nqKiIg4fPkx1dTVFRUXCkTkQCDwlIgsLC7S0tCCRSDAa\njYLohmt0h0QiEYTk1KlTFBYWcu/ePT744ANu3ryJw+EgJiaGmJgY3G43LpcLp9O57S4UJRIJSqWS\n4uJi9u7dK6wLl8vFyMgI7777Lg8fPgxbowWezEF0dLSw6ezYsUMw3IJBUEU3EAiwuLiIw+EQSBRU\nYwAAEi5JREFUfqZUKsnOzuYb3/gGk5OTLC0t4Xa7P/O7oqOjSUlJITMzc9uEuBQWFvL1r3+dv/zL\nv0QulyOTyQTL9uNIJBLh5x/993BFLpdTWVnJK6+8wr59+/jVr37FuXPnaG9vF3xzRqORkpISABYX\nF7HZbNvO2lUoFJhMJl5++WX2798v/Hx6epru7m56e3tZWloK4Qg3HrlcTnV1NUePHqW2thatVhvU\n3x9UtQoEAnR3d1NfX8/Vq1dxuVzCrlNdXc3OnTtJTk7+XN/ldrux2+0sLi4KPjm73U5LS0vYtW03\nGAzs2rWLs2fPcujQIdRqNQArKyvMz8+ztLTE6uqqYO16vV5WV1dxuVz4/X7h5+EqMHK5nKioKI4d\nO0ZycjKtra1cunSJwcFBod22TqcjJyeHgwcPEhUVFdbz8WkkJiby/e9/n4MHD2I2mwkEArjdbrq7\nu2lqasLhcIS19R8XF8eBAwd488032bNnDwqFgqmpqaBuNEF3L0xNTXHnzh2ioqLIzc0lISEBjUZD\nUlISu3fvZnR0lPHx8c8VffDxF2d1dZVHjx5tqD8m2BgMBoqKivj617/OoUOHyM7Oxu/3s7i4SF9f\nHwMDA6SmplJYWCj4pUZHR+nt7WV8fPwpXxUQlkKTlJTE/v372bFjBzabjffff5+enh4WFxeFz6wf\np4uKirBYLGG3MX9ejEYjJ06cIC0tDa1Wi8fjYWRkhObmZtrb24VNaiuz7rqMjIxEo9EQERGBTqfD\nZDKRlZVFRUUF+/fvRyaT0draSn9/P1VVVZSXlwdlfEEX3UAgwMjICNevX+f06dPo9XqioqKQyWTU\n1NTQ3d3Nhx9+yNLS0jN/+VKpVIhUMBgMQgZJuCKTycjKyuLYsWN873vfQ6lU4vf78Xg8jI6O8tZb\nb/HOO+9w+PBhvvvd75KYmIjP56OxsZELFy7Q3t6OXq/H6/Xi8/nC0g2jUCiorKzkn/7pn3C5XNTX\n1/POO+/gcDgIBAJIJBKkUinHjx/n9OnTREVFce3aNYaHh0M99KAjk8mIiIggKSkJrVaL3+/H6XRy\n8+ZNbt++zejoaKiH+EJQKBQYDAZycnJISUkhNTWVnJwcampqyMjIQKVS4fP5aGho4O2336ajo4Mf\n/OAH4Su6AB6PB5vNxsDAACkpKUL6rtlsJjExkaioKJaXl5/KBpHJZOTn5wuhHRUVFezduzcUww8K\nWq2WyspKXnvtNY4ePYpCocDr9bK4uCgI7t27d4XPW61W2tvb6ejo4MKFC3R2duL1eunt7aWjo4Od\nO3eSkpISwifaGHJzc9mxYwcxMTH87Gc/o6mpiZWVFeGIbDAY2LFjBzt27CAuLg6Hw8GNGze+VIji\nVqeiooLTp0+jUqmAJ37trq4uzp07R39/f4hH9+JQq9VkZGTw3e9+l4KCAkwmE1qtFpVKxczMDL29\nvdy8eZP29nZGR0eDboyERHQDgQBOp5OhoSEqKyuBJ0cCjUZDXFwciYmJTE1NodPpMBqNGI1GlEol\ntbW1wm1rWlraUyLidDqx2Wy43e6wKOqi0+k4fPgwe/fuFS4LHz58SEtLC7dv3+bu3buMj4/j9/ux\nWCw0NjbS3NxMb28v/f39WK1WAoEA8/PzzM7OYrfbw2JePk5hYSG5ubmsrKzQ0dHB4OCg8JwKhYLU\n1FTOnDlDfn4+brebzs5O7t+/v2EVpDYjcrmcsrIyjh07Rm1tLXK5nOXlZbq6unjnnXfo7u7GZrOF\nepgvjHXjpLOzE4fDQXR0NPBEI9azEXt6epiensbj8WAymZ46Vctksg2N/Q9ZnK7b7aa/v5/p6Wny\n8vLQaDQAJCQkUFFRgd1uJz4+nuzsbDIyMlCr1dTU1FBdXf3c73O5XDgcjk+8zd9KyGQyoqKi2Ldv\nH1lZWXi9Xh4/fsyVK1f4wx/+QENDg/BZuVzO6Ogow8PDOJ1O7Hb7U9/l8XjweDyCEG31ufk4BQUF\npKSkMDIywsjIiOCrlUgkxMfHU1FRwWuvvYZWq6Wvr4+LFy9uq2p0UqkUvV7PyZMnOXz4MPn5+Uil\nUiYnJ7l58ya//e1vWV5eDqvLM6fTyeDgIGNjYxiNRvR6PfBEc5aXl5/aYORyOX6/H5/PJ7jgIiIi\nhDDUjSBkoutwOGhsbCQnJ4e4uDjB4q2srCQ7O5u//du/RaFQoFQqhcpHnxbaERUVRXZ2NjU1Ndy4\ncYPJyclgPcoLx2AwkJqaik6nQ6FQ0NPTw9/8zd9gsVieEVWv1ytYbZ/24oRr2Fh6ejoajYaGhgYh\nX3791HTs2DG+/e1vExMTQ2trK7///e85f/78l671sRXR6/Xk5eVRVVVFTk4OOp0OiURCZ2cn7e3t\n2O32sBLcj+LxeFhYWBAuVAOBwDOnvfXojeXlZZaXlzEYDGRnZ5Odnc2VK1c2ZFwhE12/38/y8jKt\nra1kZ2cLoqvVap8rruuC8Uk3q+vVggoLC2lvb9+SorsePldXV8err76K2WxGoVCwsrLC4ODgJwas\nf1oaq1wuR6FQhF2qtEajISEhgdjYWJaXl2lubmZpaYno6GiysrLYv38/hw8fJi8vj+XlZW7cuEFD\nQwPT09PbKu3XZDKxZ88eMjMzMRgM+P1+rFYr3d3dDAwMhK3grvNZf9frl4lWqxWbzYZer2dpaWlD\n3S0hL2I+Njb2mTfJgUCA1dVV/H4/Xq9X8E9qtVpiY2OFzymVShISEoQ41q2GTCajsLCQo0ePcuLE\nCSGq48uynqW1XtBjPXMtHFAqlZjNZuFSSKPRUFxcTExMDOXl5bz66qukpaUJl4l3796lt7c37Ash\nfZTo6GgKCws5cOAA8fHxKBQKFhYWuHXrFu3t7duioM1nsa4tDoeDlZUVAoEAFosFi8WyYb8z5KL7\nefD7/czPz7O6uorNZqOtrQ273U5BQQEnTpwI9fBeGAqFglOnTnHw4EGMRuNXdgfIZDKKi4spLy8n\nPT09rELGvF4vS0tLuFwuSktL+dGPfoRKpUKn0wlHaKVSicVi4dKlS4yOjn6pSnZbmfLycl5++WUO\nHjyIQqEAnnQR+fGPf8zw8PBTmaEif2RlZWVD06A3pej6fD7BzzI6OkpLSwttbW0sLS2xtrbGwsIC\nUVFRwkIKF9aFYr1wDUB7ezvXrl37wkfi9eP3qVOnqKqqeioFOxyO1263m4mJCS5evIjT6SQjIwO7\n3U5XVxfLy8scPHgQmUxGe3s7ly5d2lZWnUwmQ6/XU1VVRVlZGUqlkkAgwPDwMB9++CETExM4nc5Q\nD3NTEggEWFlZ2dD5Cbnorq6usrCwwOjoKBqNRgj3GB4eZnJyUgiT6u/vf2pnTktLeyrjCJ4cOePj\n44mIiNiwqu/BZnx8nP7+/i/0LBqNhuzsbA4cOMDevXtJTk7GZrNx69YtBgcHcbvdW97N4PP5sNvt\nNDU1YbVaKSgowOFw4Ha7iYqKwufzMTw8TGNjI/39/dtGZJRKJXFxcdTW1lJXV0dGRgZ+v5/x8XGu\nX7/OpUuXwjZ88LNQq9UolUrk8ieyt36xtl7lcHR0lMTERLxeL1KpFJVKhVQqFZKw5ubmWFhY+Mon\nhJCLrt1uZ2BggKtXr2I2m1lZWaG/v58PPviA4eHhL5QTHRkZSWlpKfHx8UJN2a3Ouh/78yCVSlEo\nFCQmJnLgwAH+4R/+gfj4eDweD0NDQ/y///f/aGtrC6sKUg8ePBA6QEilUg4fPsyf/dmfYTAY6Orq\n4vLly7hcri2/yXxeIiMjKSkp4Yc//CF5eXno9XpcLhc3b97k7bfffirccDuwnpEok8kwm83ExsYK\nHSI8Hg9utxuHw8HMzAz37t0jNzcXlUpFfHw8NpsNlUrFnj172LVrFzdv3hTaXX0VQi66a2tr3L9/\nn3//939HqVTi8/lYWVlhYWHhC9dQUCgUGI1GNBrNlvZffjS8a9++fTidThoaGj4zSiErK4uzZ89S\nUVFBfn4+8fHxOBwOrl27xltvvUVnZ+czIWfhglQqJTMzk9raWiorK4V+cJOTk2F/Q/9RdDodSUlJ\nQqovPPF/9/X1bcmInq/CekRTWloa1dXVQhTH+uWrz+djbW0Nl8uFXq/HZDJhNpt544032L9/P/Pz\n8ygUCuLi4lCr1SwtLTEwMLD1Rdfv97O0tPSFq/x4PB5cLhcrKyuo1WqhzKFKpaK4uJj79+/T0dGx\npSwcn89Hf38/hYWFJCcnC3/hlZWVnDlz5qldeb0+bEpKilDgIyMjg9OnT5Oeno7H46G1tZWHDx9y\n7do1oZ5FuB4rFQoFL730Ert27UIikXD16lV6e3vDqvjR50GtVhMVFUVERARyuZyVlRUmJiYYGhpi\nfn4+1MMLCutV58rKyigsLCQ/P5+8vDzy8/OJi4sTmuJ6vV7W1tawWq04nU4WFxefiuldd1O6XC7m\n5+eZnJx8IZePIRfdL4vb7cZisTA8PExGRsZT7cX37NnD2NgY3d3dW+rSyOv1cvv2bcxmM2lpaSQl\nJaFSqUhJSeH73/8+Pp+P6elpenp6sFgsQhEPiUSCTCYjMjKSxMREHA4HHR0dQheAsbGxsK6qtd5c\n8MSJE+Tm5tLb20tTUxOPHj0K9dCCjl6vx2w2C+/C4uIiHR0djI+Pb5tohcjISHJzc/n2t7/Nvn37\nyMzMBHgqCWJubk7oRj41NcXjx48/9R1xOBzCyemrsmVFd2lpiVu3biGVSvn7v/97cnNzheNUaWkp\nBQUFyOVyfD7flrF2fT4fQ0NDnDt3Drvdzl//9V+TkpJCREQEBQUFBAIBcnNzqampwev1olQq0Wq1\nwgvmcDjo7++nubmZxsZGbty4ISyscEan05GdnU1sbCxjY2NcvHiR+fn5LbXhvihKSko4fvy4EKs+\nNTXFlStXmJ6eDvt1sE5ycjL79u1j3759T9XnHh4e5s6dO1y/fp2BgQGWlpbwer14vd6nUuWfh8/n\nw+l0vpA53LKi6/P5mJmZ4ebNm0RHR3P69Gmqq6tRq9VotVpMJhOJiYk8fvx4Sy221dVVRkZGhMVQ\nVlZGSUkJZWVlQpfb9VxyeOKecbvdzM3N0dfXx/nz5xkYGGB0dPSZ6I5wRCqVkp2dzRtvvIFOp6On\np4fbt29vG6tunfUmi2azGZPJhEwmo7e39ynX0lYxPr4qy8vLTE5OYrfbmZycxGq1MjIywu3bt+no\n6GB4eJiFhQWh8H+w2bKiC0+K3IyPj/Pee++h1WqJjo4mPT2dlZUVlpeXn7ICtxIOh4OHDx8KroT9\n+/cLN6o6nQ6lUsnq6ipLS0vMz89jtVoZHBzk7t27vPfee8+UxQxnjEYjpaWlnDhxgsnJSdra2hgY\nGNhSG+2LQCKRkJiYiMlkEi6KBgYGaG1t3Xa1g+fm5ujo6ODKlStoNBqhFVFbWxvT09OhHt7WFl14\nYvFOTExw+fJlpFIpr732Gj09PTQ0NDA2NrZlXz6fz4fNZuP27duMjY0xOTnJyZMnqaysJC4ujrm5\nOW7fvk1DQwMzMzMMDAwIVv12sWjgyXF6165dmM1mfvrTn1JfX7/tMs/gicWfmppKQkICKpUKiUQi\nVM7abjidTvr7+/mXf/kXAKGK2GZxN2150YUnkzo0NMTvfvc7WlpasNlsTE1NCT3Ctirr/c4sFgvX\nrl3j4cOHxMbGolarcTqdzM7OMj09jcvlYnl5eVuKTXl5OXl5eUxOTjIyMrJtbug/js/no6uri4GB\nAex2O0qlktHR0W15mQh/9MFuRsJCdAFsNhs2m4379++HeigvHLfbzdjYGGNjY6EeyqYjJSUFrVZL\ne3s7MzMzn6uTdDgSCASYmpri1q1bGI1G5HI5t2/f3naxuVuBsBFdke2JRCJhZmaGq1evMj8/v61c\nK8/j8uXLXL58OdTDEPkUJBu9SCUSyaZ9CwKBQEiqem/mOYHQzMuXnZPc3Fw0Gg2zs7MsLCxsmA9f\nXCvPIs7J8/mseRFFNwRs5jmBrSW6wUJcK88izsnz+ax52XrxVCIiIiJbmA23dEVERERE/oho6YqI\niIgEEVF0RURERIKIKLoiIiIiQUQUXREREZEgIoquiIiISBARRVdEREQkiIiiKyIiIhJERNEVERER\nCSKi6IqIiIgEEVF0RURERIKIKLoiIiIiQUQUXREREZEgIoquiIiISBARRVdEREQkiIiiKyIiIhJE\nRNEVERERCSKi6IqIiIgEEVF0RURERIKIKLoiIiIiQUQUXREREZEg8v8BfJBhGY7WT/gAAAAASUVO\nRK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7fa3926cb5d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "for i in range(25):\n",
    "    plt.subplot(5, 5, i + 1)\n",
    "    plt.imshow(X_train[i], cmap='gray')\n",
    "    plt.axis('off')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## First, prepare data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "n_features = X_train[0].shape[0] * X_train[0].shape[1]\n",
    "X_train = X_train.reshape(-1, n_features).astype('float32')\n",
    "X_test = X_test.reshape(-1, n_features).astype('float32')\n",
    "# proper scaling of input\n",
    "X_train /= 255\n",
    "X_test /= 255"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(60000, 10)\n",
      "(10000, 10)\n"
     ]
    }
   ],
   "source": [
    "# one-hot encoding of target\n",
    "n_classes = 10\n",
    "Y_train = np_utils.to_categorical(y_train, n_classes)\n",
    "Y_test = np_utils.to_categorical(y_test, n_classes)\n",
    "print Y_train.shape\n",
    "print Y_test.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train a neural network"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 60000 samples, validate on 10000 samples\n",
      "Epoch 1/5\n",
      "1s - loss: 0.3721 - acc: 0.8951 - val_loss: 0.2013 - val_acc: 0.9424\n",
      "Epoch 2/5\n",
      "1s - loss: 0.1854 - acc: 0.9465 - val_loss: 0.1418 - val_acc: 0.9575\n",
      "Epoch 3/5\n",
      "1s - loss: 0.1391 - acc: 0.9596 - val_loss: 0.1145 - val_acc: 0.9670\n",
      "Epoch 4/5\n",
      "1s - loss: 0.1146 - acc: 0.9657 - val_loss: 0.1041 - val_acc: 0.9692\n",
      "Epoch 5/5\n",
      "1s - loss: 0.0983 - acc: 0.9700 - val_loss: 0.0966 - val_acc: 0.9708\n"
     ]
    }
   ],
   "source": [
    "model = Sequential()\n",
    "model.add(Dense(output_dim=100, input_shape=(n_features,)))\n",
    "model.add(Activation('relu'))\n",
    "model.add(Dropout(p=0.1))\n",
    "model.add(Dense(output_dim=10))\n",
    "model.add(Activation('softmax'))\n",
    "\n",
    "# here keras creates all necessary variables \n",
    "# and compiles functions\n",
    "model.compile(\n",
    "    loss='categorical_crossentropy', # same as logloss, but multiclass\n",
    "    optimizer=RMSprop(), # different optimization algorithms can be used\n",
    "    metrics=['accuracy'] # which things to monitor\n",
    ")\n",
    "\n",
    "# training loop is hidden in this function\n",
    "history = model.fit(X_train, Y_train,\n",
    "                    # in practice, the 'step' uses not all the data, \n",
    "                    # but some reasonable chunk\n",
    "                    batch_size=128, \n",
    "                    # 1 epoch = 'used whole sample for updates'\n",
    "                    nb_epoch=5,\n",
    "                    verbose=2, \n",
    "                    # to monitor test quality, we pass this to the model\n",
    "                    validation_data=(X_test, Y_test)\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Checking predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train loss:  0.0713785262014\n",
      "test  loss:  0.0966423034323\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import log_loss, accuracy_score\n",
    "\n",
    "print 'train loss: ', log_loss(Y_train, model.predict_proba(X_train, verbose=0))\n",
    "print 'test  loss: ', log_loss(Y_test, model.predict_proba(X_test, verbose=0))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train accuracy:  0.979483333333\n",
      "test  accuracy:  0.9708\n"
     ]
    }
   ],
   "source": [
    "# keras looks very familiar, but it isn't scikit-learn, so .predict works differently.\n",
    "print 'train accuracy: ', accuracy_score(y_train, model.predict_proba(X_train, verbose=0).argmax(axis=1))\n",
    "print 'test  accuracy: ', accuracy_score(y_test, model.predict_proba(X_test, verbose=0).argmax(axis=1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## Optional exercises\n",
    "\n",
    "1. fine-tune the network above and look at misclassified images. For tuning, you can use the [this approach](http://i.imgur.com/bSiNEq6.gif)\n",
    "\n",
    "2. compare two optimizers: `keras.optimizers.SGD` (simple stochastic SGD) and `keras.optimizers.Adadelta` (AdaDelta) on this dataset and study their performance depending on the number of hidden layers.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# References\n",
    "- getting started with [sequential models in keras](https://keras.io/getting-started/sequential-model-guide/)\n",
    "- neural networks [playground](http://playground.tensorflow.org)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}