tutorials/2-multi-step.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Adapters: Creating steps with multiple inputs\n",
    "\n",
    "This notebook shows how to create a more complex pipeline, including steps with multiple inputs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%load_ext autoreload\n",
    "%autoreload 2\n",
    "\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "from steppy.base import Step, BaseTransformer\n",
    "from steppy.adapter import Adapter, E\n",
    "\n",
    "EXPERIMENT_DIR = './ex2'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import shutil\n",
    "\n",
    "# By default pipelines will try to load previously trained models so we delete the cache to ba sure we're starting from scratch\n",
    "shutil.rmtree(EXPERIMENT_DIR, ignore_errors=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As before, we'll import a dataset from Scikit-learn for our experiments and divide it into training and test sets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.datasets import load_breast_cancer\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "dset = load_breast_cancer()\n",
    "X_dset, y_dset = dset.data, dset.target\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(X_dset, y_dset, test_size=0.2, stratify=y_dset, random_state=42)\n",
    "\n",
    "print('{} samples for training'.format(len(y_train)))\n",
    "print('{} samples for test'.format(len(y_test)))\n",
    "\n",
    "data_train = {'input':\n",
    "                {\n",
    "                     'X': X_train,\n",
    "                     'y': y_train,\n",
    "                }\n",
    "            }\n",
    "\n",
    "data_test = {'input':\n",
    "                {\n",
    "                     'X': X_test,\n",
    "                     'y': y_test,\n",
    "                }\n",
    "            }"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creating pipeline components\n",
    "\n",
    "This time we want to build a more fancy pipeline. We'll normalize our data, run PCA to compute some features of a different flavour and then combine them with our original features in a final logistic regression step."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our first step will be a normalization step. We could use the one from Scikit-learn but we'll write a pure Numpy implementation just to show how this could be done:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.externals import joblib\n",
    "\n",
    "class NormalizationTransformer(BaseTransformer):\n",
    "    def __init__(self):\n",
    "        self.mean = None\n",
    "        self.std = None\n",
    "    \n",
    "    # Having only X as input ensures that we don't accidentally fit y\n",
    "    def fit(self, X):\n",
    "        self.mean = np.mean(X, axis=0)\n",
    "        self.std = np.std(X, axis=0)\n",
    "        return self\n",
    "\n",
    "    def transform(self, X, **kwargs):\n",
    "        X_tfm  = (X - self.mean) / self.std\n",
    "        return {'X': X_tfm}\n",
    "    \n",
    "    def persist(self, filepath):\n",
    "        joblib.dump([self.mean, self.std], filepath)\n",
    "        \n",
    "    def load(self, filepath):\n",
    "        self.mean, self.std = joblib.load(filepath)\n",
    "        return self"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll also construct a PCA transformer for our normalized features:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.decomposition import PCA\n",
    "\n",
    "class PCATransformer(BaseTransformer):\n",
    "    def __init__(self):\n",
    "        self.estimator = PCA(n_components=10)\n",
    "        \n",
    "    def fit(self, X):\n",
    "        self.estimator.fit(X)\n",
    "        return self\n",
    "\n",
    "    def transform(self, X, **kwargs):\n",
    "        X_tfm  = self.estimator.transform(X)\n",
    "        return {'X': X_tfm}\n",
    "    \n",
    "    def persist(self, filepath):\n",
    "        joblib.dump(self.estimator, filepath)\n",
    "        \n",
    "    def load(self, filepath):\n",
    "        self.estimator = joblib.load(filepath)\n",
    "        return self"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, we'll use logistic regression as our classifier:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "\n",
    "class LogRegTransformer(BaseTransformer):\n",
    "    def __init__(self):\n",
    "        self.estimator = LogisticRegression()\n",
    "        \n",
    "    def fit(self, X, y):\n",
    "        self.estimator.fit(X, y)\n",
    "        return self\n",
    "\n",
    "    def transform(self, X, **kwargs):\n",
    "        y_pred  = self.estimator.predict(X)\n",
    "        return {'y_pred': y_pred}\n",
    "    \n",
    "    def persist(self, filepath):\n",
    "        joblib.dump(self.estimator, filepath)\n",
    "        \n",
    "    def load(self, filepath):\n",
    "        self.estimator = joblib.load(filepath)\n",
    "        return self"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Assembling the pipeline\n",
    "Now we'll create steps from our transformers and link them all together:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our normalization step will only require the features from the input, not the labels. In fact, we would like to *avoid* giving it the labels just in case there could be data leak in the implementation (the first rule of data science is you don't trust anyone). To achieve this, we will use a special `adapter` argument to the step constructors, which allows us to extract just the required variables from the data dictionary."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "norm_step = Step(name='Normalizer',\n",
    "                 transformer=NormalizationTransformer(),\n",
    "                 input_data=['input'],\n",
    "                 adapter=Adapter({\n",
    "                     'X': E('input', 'X')\n",
    "                 }),\n",
    "                 experiment_directory=EXPERIMENT_DIR,\n",
    "                 is_fittable=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The notation `E('input', 'X')` tells steppy that this is a placeholder for extracting the output `X` from input data called `input`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pca_step = Step(name='PCA',\n",
    "                transformer=PCATransformer(),\n",
    "                input_steps=[norm_step],\n",
    "                experiment_directory=EXPERIMENT_DIR,\n",
    "                is_fittable=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our classifier step will have to combine two data flows: the features processed by PCA, and the labels fed directly from input. Therefore, we will have to use the `adapter` argument to specify how to map those inputs to transformer arguments."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "lr_step = Step(name='LogReg',\n",
    "               transformer=LogRegTransformer(),\n",
    "               input_steps=[pca_step],\n",
    "               input_data=['input'],\n",
    "               adapter=Adapter({\n",
    "                   'X': E('PCA', 'X'),\n",
    "                   'y': E('input', 'y')\n",
    "               }),\n",
    "               experiment_directory=EXPERIMENT_DIR,\n",
    "               is_fittable=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One may think it's a bit cumbersome to create your transformers and then have to wrap them with steps. However, there is an advantage to this - think about it:\n",
    "* The **transformer** is the ***implementation*** of a machine learning algorithm. It has an input and outputs but it doesn't even know what these are connected to.\n",
    "* The **steps** define the ***connections*** between different transformers. At this level of abstraction, all the algorithmic details are hidden. The code that defines steps and connects them together is compact and it's easier to see what is connected to what."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So what does our pipeline look like?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "lr_step"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This looks about right - let's move on to training!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Training\n",
    "\n",
    "Training a pipeline is a one-liner. When we fit the final logistic regression step, it will go back to its input steps and fit them too (assuming there's no cache or persistent outputs - that's why we delete any leftover cache at the start of the notebook). This also works recursively, so the parent steps will ask the grandparent steps to fit etc."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "preds_train = lr_step.fit_transform(data_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's see how well we do on our training data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import accuracy_score\n",
    "acc_train = accuracy_score(data_train['input']['y'], preds_train['y_pred'])\n",
    "print('Training accuracy = {:.4f}'.format(acc_train))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Generating test predictions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Running test data through our pipeline is a one-liner too:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "preds_test = lr_step.transform(data_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What is our test score?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "acc_test = accuracy_score(data_test['input']['y'], preds_test['y_pred'])\n",
    "print('Test accuracy = {:.4f}'.format(acc_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That seems pretty good. Have a look at the next notebook for even more complex pipelines with parallel branches."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}