scikit-learn-1.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# An Introduction to Machine Learning with scikit-learn\n",
    "\n",
    "From [this tutorial](http://scikit-learn.org/stable/tutorial/basic/tutorial.html).\n",
    "\n",
    "> In this section, we introduce the machine learning vocabulary that we use throughout scikit-learn and give a simple learning example."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Machine Learning: The Problem Setting\n",
    "\n",
    "__ < __ _insert typical machine learning explanatory preamble here_ __ > __\n",
    "\n",
    "![Choose Wisely](./img/ml_map.jpg)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Loading an Example Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn import datasets\n",
    "\n",
    "iris = datasets.load_iris()\n",
    "digits = datasets.load_digits()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[  0.   0.   5. ...,   0.   0.   0.]\n",
      " [  0.   0.   0. ...,  10.   0.   0.]\n",
      " [  0.   0.   0. ...,  16.   9.   0.]\n",
      " ..., \n",
      " [  0.   0.   1. ...,   6.   0.   0.]\n",
      " [  0.   0.   2. ...,  12.   0.   0.]\n",
      " [  0.   0.  10. ...,  12.   1.   0.]]\n"
     ]
    }
   ],
   "source": [
    "# The digits dataset\n",
    "print(digits.data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[ 5.1  3.5  1.4  0.2]\n",
      " [ 4.9  3.   1.4  0.2]\n",
      " [ 4.7  3.2  1.3  0.2]\n",
      " [ 4.6  3.1  1.5  0.2]\n",
      " [ 5.   3.6  1.4  0.2]\n",
      " [ 5.4  3.9  1.7  0.4]\n",
      " [ 4.6  3.4  1.4  0.3]\n",
      " [ 5.   3.4  1.5  0.2]\n",
      " [ 4.4  2.9  1.4  0.2]\n",
      " [ 4.9  3.1  1.5  0.1]\n",
      " [ 5.4  3.7  1.5  0.2]\n",
      " [ 4.8  3.4  1.6  0.2]\n",
      " [ 4.8  3.   1.4  0.1]\n",
      " [ 4.3  3.   1.1  0.1]\n",
      " [ 5.8  4.   1.2  0.2]\n",
      " [ 5.7  4.4  1.5  0.4]\n",
      " [ 5.4  3.9  1.3  0.4]\n",
      " [ 5.1  3.5  1.4  0.3]\n",
      " [ 5.7  3.8  1.7  0.3]\n",
      " [ 5.1  3.8  1.5  0.3]\n",
      " [ 5.4  3.4  1.7  0.2]\n",
      " [ 5.1  3.7  1.5  0.4]\n",
      " [ 4.6  3.6  1.   0.2]\n",
      " [ 5.1  3.3  1.7  0.5]\n",
      " [ 4.8  3.4  1.9  0.2]\n",
      " [ 5.   3.   1.6  0.2]\n",
      " [ 5.   3.4  1.6  0.4]\n",
      " [ 5.2  3.5  1.5  0.2]\n",
      " [ 5.2  3.4  1.4  0.2]\n",
      " [ 4.7  3.2  1.6  0.2]\n",
      " [ 4.8  3.1  1.6  0.2]\n",
      " [ 5.4  3.4  1.5  0.4]\n",
      " [ 5.2  4.1  1.5  0.1]\n",
      " [ 5.5  4.2  1.4  0.2]\n",
      " [ 4.9  3.1  1.5  0.1]\n",
      " [ 5.   3.2  1.2  0.2]\n",
      " [ 5.5  3.5  1.3  0.2]\n",
      " [ 4.9  3.1  1.5  0.1]\n",
      " [ 4.4  3.   1.3  0.2]\n",
      " [ 5.1  3.4  1.5  0.2]\n",
      " [ 5.   3.5  1.3  0.3]\n",
      " [ 4.5  2.3  1.3  0.3]\n",
      " [ 4.4  3.2  1.3  0.2]\n",
      " [ 5.   3.5  1.6  0.6]\n",
      " [ 5.1  3.8  1.9  0.4]\n",
      " [ 4.8  3.   1.4  0.3]\n",
      " [ 5.1  3.8  1.6  0.2]\n",
      " [ 4.6  3.2  1.4  0.2]\n",
      " [ 5.3  3.7  1.5  0.2]\n",
      " [ 5.   3.3  1.4  0.2]\n",
      " [ 7.   3.2  4.7  1.4]\n",
      " [ 6.4  3.2  4.5  1.5]\n",
      " [ 6.9  3.1  4.9  1.5]\n",
      " [ 5.5  2.3  4.   1.3]\n",
      " [ 6.5  2.8  4.6  1.5]\n",
      " [ 5.7  2.8  4.5  1.3]\n",
      " [ 6.3  3.3  4.7  1.6]\n",
      " [ 4.9  2.4  3.3  1. ]\n",
      " [ 6.6  2.9  4.6  1.3]\n",
      " [ 5.2  2.7  3.9  1.4]\n",
      " [ 5.   2.   3.5  1. ]\n",
      " [ 5.9  3.   4.2  1.5]\n",
      " [ 6.   2.2  4.   1. ]\n",
      " [ 6.1  2.9  4.7  1.4]\n",
      " [ 5.6  2.9  3.6  1.3]\n",
      " [ 6.7  3.1  4.4  1.4]\n",
      " [ 5.6  3.   4.5  1.5]\n",
      " [ 5.8  2.7  4.1  1. ]\n",
      " [ 6.2  2.2  4.5  1.5]\n",
      " [ 5.6  2.5  3.9  1.1]\n",
      " [ 5.9  3.2  4.8  1.8]\n",
      " [ 6.1  2.8  4.   1.3]\n",
      " [ 6.3  2.5  4.9  1.5]\n",
      " [ 6.1  2.8  4.7  1.2]\n",
      " [ 6.4  2.9  4.3  1.3]\n",
      " [ 6.6  3.   4.4  1.4]\n",
      " [ 6.8  2.8  4.8  1.4]\n",
      " [ 6.7  3.   5.   1.7]\n",
      " [ 6.   2.9  4.5  1.5]\n",
      " [ 5.7  2.6  3.5  1. ]\n",
      " [ 5.5  2.4  3.8  1.1]\n",
      " [ 5.5  2.4  3.7  1. ]\n",
      " [ 5.8  2.7  3.9  1.2]\n",
      " [ 6.   2.7  5.1  1.6]\n",
      " [ 5.4  3.   4.5  1.5]\n",
      " [ 6.   3.4  4.5  1.6]\n",
      " [ 6.7  3.1  4.7  1.5]\n",
      " [ 6.3  2.3  4.4  1.3]\n",
      " [ 5.6  3.   4.1  1.3]\n",
      " [ 5.5  2.5  4.   1.3]\n",
      " [ 5.5  2.6  4.4  1.2]\n",
      " [ 6.1  3.   4.6  1.4]\n",
      " [ 5.8  2.6  4.   1.2]\n",
      " [ 5.   2.3  3.3  1. ]\n",
      " [ 5.6  2.7  4.2  1.3]\n",
      " [ 5.7  3.   4.2  1.2]\n",
      " [ 5.7  2.9  4.2  1.3]\n",
      " [ 6.2  2.9  4.3  1.3]\n",
      " [ 5.1  2.5  3.   1.1]\n",
      " [ 5.7  2.8  4.1  1.3]\n",
      " [ 6.3  3.3  6.   2.5]\n",
      " [ 5.8  2.7  5.1  1.9]\n",
      " [ 7.1  3.   5.9  2.1]\n",
      " [ 6.3  2.9  5.6  1.8]\n",
      " [ 6.5  3.   5.8  2.2]\n",
      " [ 7.6  3.   6.6  2.1]\n",
      " [ 4.9  2.5  4.5  1.7]\n",
      " [ 7.3  2.9  6.3  1.8]\n",
      " [ 6.7  2.5  5.8  1.8]\n",
      " [ 7.2  3.6  6.1  2.5]\n",
      " [ 6.5  3.2  5.1  2. ]\n",
      " [ 6.4  2.7  5.3  1.9]\n",
      " [ 6.8  3.   5.5  2.1]\n",
      " [ 5.7  2.5  5.   2. ]\n",
      " [ 5.8  2.8  5.1  2.4]\n",
      " [ 6.4  3.2  5.3  2.3]\n",
      " [ 6.5  3.   5.5  1.8]\n",
      " [ 7.7  3.8  6.7  2.2]\n",
      " [ 7.7  2.6  6.9  2.3]\n",
      " [ 6.   2.2  5.   1.5]\n",
      " [ 6.9  3.2  5.7  2.3]\n",
      " [ 5.6  2.8  4.9  2. ]\n",
      " [ 7.7  2.8  6.7  2. ]\n",
      " [ 6.3  2.7  4.9  1.8]\n",
      " [ 6.7  3.3  5.7  2.1]\n",
      " [ 7.2  3.2  6.   1.8]\n",
      " [ 6.2  2.8  4.8  1.8]\n",
      " [ 6.1  3.   4.9  1.8]\n",
      " [ 6.4  2.8  5.6  2.1]\n",
      " [ 7.2  3.   5.8  1.6]\n",
      " [ 7.4  2.8  6.1  1.9]\n",
      " [ 7.9  3.8  6.4  2. ]\n",
      " [ 6.4  2.8  5.6  2.2]\n",
      " [ 6.3  2.8  5.1  1.5]\n",
      " [ 6.1  2.6  5.6  1.4]\n",
      " [ 7.7  3.   6.1  2.3]\n",
      " [ 6.3  3.4  5.6  2.4]\n",
      " [ 6.4  3.1  5.5  1.8]\n",
      " [ 6.   3.   4.8  1.8]\n",
      " [ 6.9  3.1  5.4  2.1]\n",
      " [ 6.7  3.1  5.6  2.4]\n",
      " [ 6.9  3.1  5.1  2.3]\n",
      " [ 5.8  2.7  5.1  1.9]\n",
      " [ 6.8  3.2  5.9  2.3]\n",
      " [ 6.7  3.3  5.7  2.5]\n",
      " [ 6.7  3.   5.2  2.3]\n",
      " [ 6.3  2.5  5.   1.9]\n",
      " [ 6.5  3.   5.2  2. ]\n",
      " [ 6.2  3.4  5.4  2.3]\n",
      " [ 5.9  3.   5.1  1.8]]\n"
     ]
    }
   ],
   "source": [
    "# The iris dataset\n",
    "print(iris.data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0, 1, 2, ..., 8, 9, 8])"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# The digits dataset class attribute possible values (our classification labels)\n",
    "digits.target"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[  0.,   0.,   5.,  13.,   9.,   1.,   0.,   0.],\n",
       "       [  0.,   0.,  13.,  15.,  10.,  15.,   5.,   0.],\n",
       "       [  0.,   3.,  15.,   2.,   0.,  11.,   8.,   0.],\n",
       "       [  0.,   4.,  12.,   0.,   0.,   8.,   8.,   0.],\n",
       "       [  0.,   5.,   8.,   0.,   0.,   9.,   8.,   0.],\n",
       "       [  0.,   4.,  11.,   0.,   1.,  12.,   7.,   0.],\n",
       "       [  0.,   2.,  14.,   5.,  10.,  12.,   0.,   0.],\n",
       "       [  0.,   0.,   6.,  13.,  10.,   0.,   0.,   0.]])"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Shape of the digits dataset (each original sample is an image of shape (8, 8))\n",
    "digits.images[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Learning and Predicting\n",
    "\n",
    "> In the case of the digits dataset, the task is to predict, given an image, which digit it represents. We are given samples of each of the 10 possible classes (the digits zero through nine) on which we fit an estimator to be able to predict the classes to which unseen samples belong.\n",
    "In scikit-learn, an estimator for classification is a Python object that implements the methods `fit(X, y)` and `predict(T)`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Implement the support vector machine estimator\n",
    "from sklearn import svm\n",
    "# clf == classifier\n",
    "clf = svm.SVC(gamma=0.001, C=100.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "SVC(C=100.0, cache_size=200, class_weight=None, coef0=0.0, degree=3,\n",
       "  gamma=0.001, kernel='rbf', max_iter=-1, probability=False,\n",
       "  random_state=None, shrinking=True, tol=0.001, verbose=False)"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Fit classifier to the model by passing our training set to the fit method\n",
    "# As a training set we'll use all the images of our dataset apart from the last one\n",
    "clf.fit(digits.data[:-1], digits.target[:-1])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([8])"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Now we predict the final digit image, based on our learned classifier\n",
    "clf.predict(digits.data[-1:])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model Persistence\n",
    "\n",
    "> It is possible to save a model in the scikit by using Python’s built-in persistence model, namely pickle."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0, degree=3, gamma=0.0,\n",
       "  kernel='rbf', max_iter=-1, probability=False, random_state=None,\n",
       "  shrinking=True, tol=0.001, verbose=False)"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Build a classifier\n",
    "\n",
    "from sklearn import svm\n",
    "from sklearn import datasets\n",
    "\n",
    "clf = svm.SVC()\n",
    "iris = datasets.load_iris()\n",
    "X, y = iris.data, iris.target\n",
    "clf.fit(X, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([0])"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Pickle it\n",
    "\n",
    "import pickle\n",
    "\n",
    "s = pickle.dumps(clf)\n",
    "clf2 = pickle.loads(s)\n",
    "clf2.predict(X[0:1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Conventions\n",
    "\n",
    "> scikit-learn estimators follow certain rules to make their behavior more predictive."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dtype('float32')"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Unless otherwise specified, input will be cast to float64\n",
    "\n",
    "import numpy as np\n",
    "from sklearn import random_projection\n",
    "\n",
    "rng = np.random.RandomState(0)\n",
    "X = rng.rand(10, 2000)\n",
    "X = np.array(X, dtype='float32')\n",
    "X.dtype"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dtype('float64')"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "transformer = random_projection.GaussianRandomProjection()\n",
    "X_new = transformer.fit_transform(X)\n",
    "X_new.dtype"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0, degree=3, gamma=0.0,\n",
       "  kernel='rbf', max_iter=-1, probability=False, random_state=None,\n",
       "  shrinking=True, tol=0.001, verbose=False)"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Regression targets are cast to float64, classification targets are maintained\n",
    "\n",
    "from sklearn import datasets\n",
    "from sklearn.svm import SVC\n",
    "\n",
    "iris = datasets.load_iris()\n",
    "clf = SVC()\n",
    "clf.fit(iris.data, iris.target)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[0, 0, 0]"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list(clf.predict(iris.data[:3]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0, degree=3, gamma=0.0,\n",
       "  kernel='rbf', max_iter=-1, probability=False, random_state=None,\n",
       "  shrinking=True, tol=0.001, verbose=False)"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "clf.fit(iris.data, iris.target_names[iris.target])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['setosa', 'setosa', 'setosa']"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list(clf.predict(iris.data[:3]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Here, the first predict() returns an integer array, since iris.target (an integer array) was used in fit. The second predict returns a string array, since iris.target_names was for fitting."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Refitting and updating parameters\n",
    "\n",
    "> Hyper-parameters of an estimator can be updated after it has been constructed via the sklearn.pipeline.Pipeline.set_params method. Calling fit() more than once will overwrite what was learned by any previous fit():"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}