Khan Academy - Linear Regression.ipynb

{
 "metadata": {
  "name": "Khan Academy - Linear Regression"
 },
 "nbformat": 3,
 "nbformat_minor": 0,
 "worksheets": [
  {
   "cells": [
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "%pylab inline\n",
      "%load_ext sympyprinting\n",
      "import sympy as sym\n",
      "import numpy as np\n",
      "from IPython.display import display, Math, Latex\n",
      "import matplotlib.pyplot as plt\n",
      "from sympy.printing.python import python\n",
      "from sympy.printing.latex import latex\n"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "\n",
        "Welcome to pylab, a matplotlib-based Python environment [backend: module://IPython.zmq.pylab.backend_inline].\n",
        "For more information, type 'help(pylab)'.\n"
       ]
      }
     ],
     "prompt_number": 75
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "#Linear Regression\n",
      "\n",
      "##Correlation and Causality\n",
      "\n",
      "##Squared Error of Regression Line\n",
      "\n",
      "$$\n",
      "m = slope\n",
      "$$\n",
      "\n",
      "$$\n",
      "x, y = 0, b\n",
      "$$\n",
      "\n",
      "$$\n",
      "y = mx + b\n",
      "$$\n",
      "\n",
      "b is the y intercept.\n",
      "\n",
      "Each point off of our estimated line has an \"error\". We want to minimize the squares of these errors.\n",
      "\n",
      "The Squared Error of the Line:\n",
      "$$\n",
      "\\begin{align*}\n",
      "SE_{line} =  (y_1 - (mx_1 + b))^2 + (y_2 - (mx_2 + b))^2 +\\cdots + (y_n - (mx_n + b))^2\n",
      "\\end{align*}\n",
      "$$\n",
      "\n"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Proof (Part 1) Minimizing Squared Error to Regression Line\n",
      "https://www.khanacademy.org/math/probability/regression/regression-correlation/v/proof--part-1--minimizing-squared-error-to-regression-line\n",
      "\n",
      "$$\n",
      "\\begin{align*}\n",
      "SE_{line} &= y_1^{2} - 2y_1(mx_1 + b) + (mx_1 + b)^2 \\\\\\\\\n",
      "&+  y_2^{2} - 2y_2(mx_2 + b) + (mx_2+ b)^2 \\\\\\\\\n",
      "&\\vdots \\\\\\\n",
      "&+  y_n^{2} - 2y_n(mx_n + b) + (mx_n+ b)^2 \\\\\\\\\n",
      "\\end{align*}\n",
      "$$\n",
      "\n",
      "$$\n",
      "\\begin{align*}\n",
      "SE_{line} &= y_1^{2} - 2y_1mx_1 + 2y_1b + m^{2}x_1^{2} + 2mx_1b + b^2 \\\\\\\\\n",
      "&+  y_2^{2} - 2y_2mx_2 + 2y_2b + m^{2}x_2^{2} + 2mx_2b + b^2 \\\\\\\\\n",
      "&\\vdots \\\\\\\n",
      "&+  y_n^{2} - 2y_nmx_n + 2y_nb + m^{2}x_n^{2} + 2mx_nb + b^2 \\\\\\\\\n",
      "\\end{align*}\n",
      "$$\n",
      "\n",
      "\n",
      "$$\n",
      "\\begin{align*}\n",
      "SE_{line} &= (y_1^2 + y_2^2 + \\cdots + y_n^2)\\\\\\\\ \n",
      "&- 2m(x_1y_1 + x_2y_2 + \\cdots + x_ny_n)\\\\\\\\\n",
      "&- 2b(y_1 + y_2 + \\cdots + y_n) \\\\\\\\\n",
      "&+ m^2(x_1^2 + x_2^2 + \\cdots + x_n^2)\\\\\\\\\n",
      "&+ 2mb(x_1 + x_2 + \\cdots + x_n) \\\\\\\\\n",
      "&+ nb^2\n",
      "\\end{align*}\n",
      "$$\n"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Proof Part 2 Minimizing Squared Error to Line\n",
      "https://www.khanacademy.org/math/probability/regression/regression-correlation/v/proof-part-2-minimizing-squared-error-to-line\n",
      "\n",
      "Remember $\\bar{x}$, the sample mean? Below, we find this is the mean of the of the values.\n",
      "\n",
      "$$\n",
      "\\frac{y_1^2 + y_2^2 + \\cdots + y_n^2} {n} = \\bar{y}^2\\\\\\\\\n",
      "y_1^2 + y_2^2 + \\cdots + y_n^2 = n\\bar{y}^2\n",
      "$$\n",
      "\n",
      "\n",
      "$$\n",
      "\\begin{align*}\n",
      "+ (y_1^2 + y_2^2 + \\cdots + y_n^2) = n\\bar{y}^2\\\\\\\\\n",
      "- 2m(x_1y_1 + x_2y_2 + \\cdots + x_ny_n) = n\\bar{xy}\\\\\\\\\n",
      "- 2b(y_1 + y_2 + \\cdots + y_n) = 2bn\\bar{y}\\\\\\\\\n",
      "+ m^2(x_1^2 + x_2^2 + \\cdots + x_n^2) = m^2n\\bar{x^2}\\\\\\\\\n",
      "+ 2mb(x_1 + x_2 + \\cdots + x_n) = 2mbn\\bar{x} \\\\\\\\\n",
      "\\end{align*}\n",
      "$$\n",
      "\n",
      "$$\n",
      "\\begin{align*}\n",
      "SE_{line} &= n\\bar{y}^2 - 2mn\\bar{xy} - 2bn\\bar{y} + m^2n\\bar{x^2} + 2mbn\\bar{x} + nb^2\n",
      "\\end{align*}\n",
      "$$\n",
      "\n",
      "Now, to minimize this expression.\n",
      "\n"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "##Proof Part 3 Minimizing Squared Error to Line\n",
      "https://www.khanacademy.org/math/probability/regression/regression-correlation/v/proof--part-3--minimizing-squared-error-to-regression-line\n",
      "\n",
      "Find the partial deriviatives of this expression with respect to $m$ and $b$.\n",
      "\n",
      "$$\n",
      "\\begin{align*}\n",
      "SE_{line} &= n\\bar{y}^2 - 2mn\\bar{xy} - 2bn\\bar{y} + m^2n\\bar{x^2} + 2mbn\\bar{x} + nb^2\n",
      "\\end{align*}\n",
      "$$\n",
      "\n",
      "\n",
      "$$\n",
      "\\frac{{\\partial}{SE}} {\\partial{m}} = 0\\\\\\\\\n",
      "\\frac{{\\partial}{SE}} {\\partial{b}} = 0\n",
      "$$\n",
      "\n",
      "Partial derivative with respect to $m$.\n",
      "$$\n",
      "- 2n\\bar{xy} + 2n\\bar{x^2}m + 2bn\\bar{x} = 0\n",
      "$$\n",
      "\n",
      "Partial derivative with respect to $b$.\n",
      "$$\n",
      "- 2n\\bar{y} + 2mn\\bar{x} + 2bn = 0\n",
      "$$\n",
      "\n",
      "Divide the top equation by 2n:\n",
      "$$- \\bar{xy} + m\\bar{x^2} + b\\bar{x} = 0$$\n",
      "\n",
      "Divide the second equation by 2n:\n",
      "$$- \\bar{y} + m\\bar{x} + b = 0$$\n",
      "\n",
      "Add the mean of the x, y values to the top equation:\n",
      "$$\\bar{xy} - \\bar{xy} + m\\bar{x^2} + b\\bar{x} = 0 + \\bar{xy}\\\\\\\\\n",
      "m\\bar{x^2} + b\\bar{x} = \\bar{xy}$$\n",
      "\n",
      "Add the mean of y value to the bottom equation:\n",
      "$$\\bar{y} - \\bar{y} + m\\bar{x} + b = 0 + \\bar{y}\\\\\\\\\n",
      "m\\bar{x} + b = \\bar{y}\n",
      "$$\n",
      "\n",
      "The point at the mean of $x$ and the mean of $y$, $(\\bar{x}, \\bar{y})$, lies on the optimal line.\n",
      "\n",
      "We can also take the first equation one step further, dividing both sides by $\\bar{x}$:\n",
      "$$\\frac {m\\bar{x^2} + b\\bar{x}} {\\bar{x}} = \\frac{\\bar{xy}} {\\bar{x}} $$\n",
      "$$m\\frac{\\bar{x^2}}{\\bar{x}} + b = \\frac{\\bar{xy}} {\\bar{x}}$$\n",
      "\n",
      "Not only does  $(\\bar{x}, \\bar{y}) $  lie on the best-fitting line, but so does $(\\frac{\\bar{x^2}} {\\bar{x}}, \\frac{\\bar{xy}} {\\bar{x}})$."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "\n",
      "$$\\begin{align*}\n",
      "m\\bar{x} + b &= \\bar{y}\\\\\\\n",
      "- m\\frac{\\bar{x^2}}{\\bar{x}} - b &= - \\frac{\\bar{xy}} {\\bar{x}}\\\\\\\\\n",
      "\\hline\\\\\\\\\n",
      "m(\\bar{x} - \\frac{\\bar{x^2}} {\\bar{x}}) &= \\bar{y} - \\frac{\\bar{xy}} {\\bar{x}}\n",
      "\\end{align*}\n",
      "$$\n",
      "\n",
      "##Proof (Part 4) Minimizing Squared Error to Regression Line\n",
      "https://www.khanacademy.org/math/probability/regression/regression-correlation/v/proof--part-4--minimizing-squared-error-to-regression-line\n",
      "\n",
      "Solve for $m$ :\n",
      "$$\\begin{align*}\n",
      "m = \\frac{\\bar{y} - \\frac{\\bar{xy}} {\\bar{x}}} {\\bar{x} - \\frac {\\bar{x^2}} {\\bar{x}}}\n",
      "\\end{align*}\n",
      "$$\n",
      "\n",
      "Multiply by $\\frac{(\\bar{x})} {(\\bar{x})}$\n",
      "\n",
      "$$\\begin{align*}\n",
      "m = \\frac{\\bar{y} - \\frac{\\bar{xy}} {\\bar{x}}} {\\bar{x} - \\frac {\\bar{x^2}} {\\bar{x}}} \\frac{(\\bar{x})} {(\\bar{x})} = \\frac {\\bar{x}\\bar{y} - \\bar{xy}} {(\\bar{x})^2 - \\bar{x^2}}\n",
      "\\end{align*}\n",
      "$$\n",
      "\n",
      "If we want to solve for $b$, we can substitute back $m\\bar{x} + b = \\bar{y}$ :\n",
      "\n",
      "$$\n",
      "b = \\bar{y} - m\\bar{x}\n",
      "$$\n"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Two key formulas!!!!\n",
      "\n",
      "### Slope\n",
      "$$\\begin{align*}\n",
      "m = \\frac {\\bar{x}\\bar{y} - \\bar{xy}} {(\\bar{x})^2 - \\bar{x^2}}\n",
      "\\end{align*}\n",
      "$$\n",
      "\n",
      "### Y Intercept\n",
      "$$\n",
      "b = \\bar{y} - m\\bar{x}\n",
      "$$"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Practice\n",
      "\n",
      "Take three points: (1, 2), (2, 1), and (4, 3), find best fitting regression line."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "x = np.array([1,2,4])\n",
      "y = np.array([2,1,3])\n",
      "# Lets make a nice chart!\n",
      "plt.figure()\n",
      "ax1 = plt.axes(frameon=True)\n",
      "ax1.get_xaxis().tick_bottom()\n",
      "ax1.get_yaxis().tick_left()\n",
      "plt.axis([0, 4.5, 0, 4.5])\n",
      "plt.plot(x, y, 'ro')\n",
      "#plt.plot(x, y, 'rx')\n",
      "plt.xlabel('x')\n",
      "plt.ylabel('y')\n",
      "plt.show()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEICAYAAABfz4NwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGE9JREFUeJzt3X9M1Pfhx/HXCabs1IEsEZODtE7Yymk9jtpdsmg9ZxzD\nH8RFY7CpEqsJIQHUP5qt1WaYWdNGtgY1IfQfF9tkNCNLepUfq01FpxSZLV2auERkEu/QkRGlihM5\nuc/3j++nN09QOPjcHbrnIyG5+3zeeq+8PT8vPp/Pfe5jMwzDEADgf96MRAcAAEwPFAIAQBKFAAAw\nUQgAAEkUAgDARCEAACTFsBBGRkbkdru1fv36UetaW1uVmpoqt9stt9utAwcOxCoGAGCCkmP1F9fU\n1MjpdOr27dtjrl+xYoV8Pl+sXh4AEKWY7CEEAgE1NTVp586detR1b1wPBwDTS0z2EPbs2aNDhw7p\n1q1bY6632Wxqa2uTy+WSw+FQdXW1nE7nqDEAgOhN9hduywvhxIkTmjdvntxut1pbW8cck5+fL7/f\nL7vdrubmZm3YsEGXLl0aNY69CACIzlR+mbZZ/V1Gb775pj744AMlJydraGhIt27d0saNG3X8+PFH\n/pkFCxboyy+/VHp6+n+D2WwUAgBEaSrbTssL4UGnT59WdXW1Pvnkk4jlfX19mjdvnmw2mzo6OrR5\n82b19PREBqMQACBqU9l2xuxTRt/5bvelrq5OklRaWqqGhgbV1tYqOTlZdrtd9fX1sY4BABhHTPcQ\npoI9BACI3lS2nVypDACQRCEAAEwUAgBAEoUAADBRCAAASRQCAMBEIQAAJFEIAAAThQAAkEQhAABM\nFAIAQBKFAAAwUQgAAEkUAgDARCEAACRRCAAAU8wKYWRkRG63W+vXrx9zfWVlpXJycuRyudTZ2Rmr\nGACACYpZIdTU1MjpdIZvofmgpqYmXb58WV1dXXr//fdVVlYWqxgAgAmKyT2VA4GAmpqatHfvXv3+\n978ftd7n86mkpESS5PF4NDAwoL6+PmVkZESMq6qqCj/2er3yer2xiAsAUIwKYc+ePTp06JBu3bo1\n5vre3l5lZWWFn2dmZioQCDy2EAAAsWX5IaMTJ05o3rx5crvdj73R88Prxjq0BACIH8sLoa2tTT6f\nTwsWLNCWLVv0+eefa9u2bRFjHA6H/H5/+HkgEJDD4bA6CgAgCpYXwsGDB+X3+3XlyhXV19frZz/7\nmY4fPx4xpqioKLysvb1daWlpow4XAQDiKybnEB703aGguro6SVJpaanWrFmjpqYmZWdna9asWTp2\n7FisYwAAxmEzHnegP4FsNttjz0EAAEabyraTK5UBAJIoBACAiUIAAEiiEAAAJgoBACCJQgAAmCgE\nAIAkCgEAYKIQAACSKAQAgIlCAABIohAAACYKAQAgiUIAAJgoBACApBgUwtDQkDwej/Ly8uR0OvXG\nG2+MGtPa2qrU1FS53W653W4dOHDA6hgAgChZfse0lJQUnTp1Sna7Xffv39eyZct09uxZLVu2LGLc\nihUr5PP5rH55AMAkxeSQkd1ulyQNDw9rZGRE6enpo8ZwNzQAmF5ick/lUCik/Px8dXd3q6ysTE6n\nM2K9zWZTW1ubXC6XHA6HqqurR42RpKqqqvBjr9crr9cbi7gAAMX4nsrffvutCgoK9M4770RszG/f\nvq2kpCTZ7XY1Nzdr165dunTpUmQw7qkMAFGbtvdUTk1N1dq1a3XhwoWI5XPmzAkfViosLFQwGNSN\nGzdiGQUAMA7LC6G/v18DAwOSpLt37+rkyZNyu90RY/r6+sIN1tHRIcMwxjzPAACIH8vPIVy/fl0l\nJSUKhUIKhULaunWrVq1apbq6OklSaWmpGhoaVFtbq+TkZNntdtXX11sdAwAQpZieQ5gKziEAQPSm\n7TkEAMCTg0IAAEiiEAAAJgoBACCJQgAAmCgEAIAkCgEAYKIQAACSKAQAgIlCAABIohAAACYKAQAg\niUIAAJgoBACAJAoBAGCiEAAAkmJQCENDQ/J4PMrLy5PT6dQbb7wx5rjKykrl5OTI5XKps7PT6hgA\nEDNnGhu1r6BAVV6v9hUU6ExjY6IjWcLyW2impKTo1KlTstvtun//vpYtW6azZ89q2bJl4TFNTU26\nfPmyurq6dP78eZWVlam9vd3qKABguTONjfrLrl16u7s7vGyv+fjltWsTFcsSMTlkZLfbJUnDw8Ma\nGRlRenp6xHqfz6eSkhJJksfj0cDAgPr6+mIRBQAs9enhwxFlIElvd3fr5JEjCUpkHcv3ECQpFAop\nPz9f3d3dKisrk9PpjFjf29urrKys8PPMzEwFAgFlZGREjKuqqgo/9nq98nq9sYgLABOWfO/emMuT\nhobinMR6MSmEGTNm6Ouvv9a3336rgoICtba2jtqYP3wTaJvNNurvebAQAGA6uP/MM2MuH0lJiXMS\n68X0U0apqalau3atLly4ELHc4XDI7/eHnwcCATkcjlhGAQBL/LyyUnsXLoxY9ubChVpdUZGgRNax\nfA+hv79fycnJSktL0927d3Xy5En95je/iRhTVFSko0ePqri4WO3t7UpLSxt1uAgApqPvThy/deSI\nkoaGNJKSol9UVDzxJ5QlyWY8fOxmir755huVlJQoFAopFApp69atev3111VXVydJKi0tlSSVl5er\npaVFs2bN0rFjx5Sfnx8ZzGYbdVgJAPB4U9l2Wl4IVqEQACB6U9l2cqUyAEAShQAAMFEIAABJFAIA\nwEQhAAAkUQgAABOFAACQRCEAAEwUAgBAEoUAADBRCAAASRQCAMBEIQAAJFEIAAAThQAAkEQhAABM\nlheC3+/XypUrtWjRIi1evFiHDx8eNaa1tVWpqalyu91yu906cOCA1TEAAFGy/J7KM2fO1Hvvvae8\nvDwNDg7qxRdf1OrVq5WbmxsxbsWKFfL5fFa/PABgkizfQ5g/f77y8vIkSbNnz1Zubq6uXbs2ahy3\nxwSA6cXyPYQH9fT0qLOzUx6PJ2K5zWZTW1ubXC6XHA6Hqqur5XQ6R/35qqqq8GOv1yuv1xvLuADw\nP81mxOhX9cHBQXm9Xu3bt08bNmyIWHf79m0lJSXJbrerublZu3bt0qVLlyKDTeFG0QDwv2oq286Y\nFEIwGNS6detUWFio3bt3jzt+wYIF+vLLL5Wenv7fYBQCAERtKttOy88hGIahHTt2yOl0PrIM+vr6\nwoE7OjpkGEZEGQAA4s/ycwjnzp3Thx9+qCVLlsjtdkuSDh48qKtXr0qSSktL1dDQoNraWiUnJ8tu\nt6u+vt7qGACAKMXsHMJUccgIAKI3rQ4ZAQCeTBQCAEAShQAAMFEIAABJFAIAwPTIQjh8+LBu3rwZ\nzywAgAR6ZCH09fXppZde0ubNm9XS0sJHQAHgKffY6xBCoZA+/fRT/eEPf9CFCxe0efNm7dixQwsX\nLox9MK5DAICoxew6hBkzZmj+/PnKyMhQUlKSbt68qU2bNun111+f1IsBAKavR+4h1NTU6Pjx4/rB\nD36gnTt36pe//KVmzpypUCiknJwcdXd3xzYYewgAELWpbDsf+V1GN27c0J///Gc9++yzEctnzJih\nTz75ZFIvBgCYvvguIwB4ivBdRgCAKaMQAACSKAQAgIlCAABIikEh+P1+rVy5UosWLdLixYt1+PDh\nMcdVVlYqJydHLpdLnZ2dVsd46p1pbNS+ggJVeb3aV1CgM42NiY4E4Aln+S00Z86cqffee095eXka\nHBzUiy++qNWrVys3Nzc8pqmpSZcvX1ZXV5fOnz+vsrIytbe3Wx3lqXWmsVF/2bVLbz9wLche8/HL\na9cmKhaAJ5zlewjz589XXl6eJGn27NnKzc3VtWvXIsb4fD6VlJRIkjwejwYGBtTX12d1lKfWp4cP\nR5SBJL3d3a2TR44kKBGAp4HlewgP6unpUWdnpzweT8Ty3t5eZWVlhZ9nZmYqEAgoIyMjYlxVVVX4\nsdfrldfrjWXcJ0byvXtjLk8aGopzEgBPk5gVwuDgoDZt2qSamhrNnj171PqHL5yw2WyjxjxYCPiv\n+888M+bykZSUOCcB8DSJyaeMgsGgNm7cqFdffVUbNmwYtd7hcMjv94efBwIBORyOWER5Kv28slJ7\nH/rG2TcXLtTqiooEJQLwNLB8D8EwDO3YsUNOp1O7d+8ec0xRUZGOHj2q4uJitbe3Ky0tbdThIjza\ndyeO3zpyRElDQxpJSdEvKio4oQxgSiz/LqOzZ8/q5Zdf1pIlS8KHgQ4ePKirV69KkkpLSyVJ5eXl\namlp0axZs3Ts2DHl5+dHBuO7jAAgalPZdvLldgDwFOHL7QAAU0YhAAAkUQgAABOFAACQRCEAAEwU\nAgBAEoUAADBRCAAASRQCAMBEIQAAJFEIAAAThQAAkEQhAABMFAIAQBKFAAAwxaQQXnvtNWVkZOiF\nF14Yc31ra6tSU1Pldrvldrt14MCBWMQAAETB8ltoStL27dtVUVGhbdu2PXLMihUr5PP5YvHyAIBJ\niMkewvLlyzV37tzHjuFuaAAwvcRkD2E8NptNbW1tcrlccjgcqq6ultPpHDWuqqoq/Njr9crr9cYv\nJAD8j4nZPZV7enq0fv16ffPNN6PW3b59W0lJSbLb7WpubtauXbt06dKlyGDcUxkAovbE3VN5zpw5\nstvtkqTCwkIFg0HduHEjEVEAAKaEFEJfX1+4wTo6OmQYhtLT0xMRBQBgisk5hC1btuj06dPq7+9X\nVlaW9u/fr2AwKEkqLS1VQ0ODamtrlZycLLvdrvr6+ljEAABEIWbnEKaKcwgAEL0n7hwCAGD6oRAA\nAJIoBACAiUIAAEiiEAAAJgoBACCJQgAAmCgEAIAkCgEAYKIQAACSKAQAgIlCAABIohAAACYKAQAg\niUIAAJgoBACApBgUwmuvvaaMjAy98MILjxxTWVmpnJwcuVwudXZ2Wh0BiHCmsVH7CgpU5fVqX0GB\nzjQ2JjoSMC1ZfgvN7du3q6KiQtu2bRtzfVNTky5fvqyuri6dP39eZWVlam9vtzoGIOn/y+Avu3bp\n7e7u8LK95uOX165NVCxgWrJ8D2H58uWaO3fuI9f7fD6VlJRIkjwejwYGBtTX12d1DECS9OnhwxFl\nIElvd3fr5JEjCUoETF+W7yGMp7e3V1lZWeHnmZmZCgQCysjIGDW2qqoq/Njr9crr9cYhIZ4myffu\njbk8aWgozkmA6S/uhSBp1A2gbTbbmOMeLARgMu4/88yYy0dSUuKcBJj+4v4pI4fDIb/fH34eCATk\ncDjiHQP/I35eWam9CxdGLHtz4UKtrqhIUCJg+or7HkJRUZGOHj2q4uJitbe3Ky0tbczDRYAVvjtx\n/NaRI0oaGtJISop+UVHBCWVgDDbj4eM3U7RlyxadPn1a/f39ysjI0P79+xUMBiVJpaWlkqTy8nK1\ntLRo1qxZOnbsmPLz80cHs9lGHVoCADzeVLadlheCVSgEAIjeVLadXKkMAJBEIQAATBQCAEAShQAA\nMFEIAABJFAIAwEQhAAAkUQgAABOFAACQRCEAAEwUAgBAEoUAADBRCAAASRQCAMBEIQAAJFEIAABT\nTAqhpaVFzz//vHJycvTuu++OWt/a2qrU1FS53W653W4dOHAgFjEAAFGw/J7KIyMjKi8v12effSaH\nw6GXXnpJRUVFys3NjRi3YsUK+Xw+q18eADBJlu8hdHR0KDs7W88995xmzpyp4uJiffzxx6PGcXtM\nAJheLN9D6O3tVVZWVvh5Zmamzp8/HzHGZrOpra1NLpdLDodD1dXVcjqdo/6uqqqq8GOv1yuv12t1\nXACAyfJCsNls447Jz8+X3++X3W5Xc3OzNmzYoEuXLo0a92AhAABiy/JDRg6HQ36/P/zc7/crMzMz\nYsycOXNkt9slSYWFhQoGg7px44bVUQAAUbC8EJYuXaquri719PRoeHhYH330kYqKiiLG9PX1hc8h\ndHR0yDAMpaenWx0FABAFyw8ZJScn6+jRoyooKNDIyIh27Nih3Nxc1dXVSZJKS0vV0NCg2tpaJScn\ny263q76+3uoYAIAo2Yxp+nEfm83GJ5EAIEpT2XZypTIAQBKFAAAwUQgAAEkUAgDARCEAACRRCAAA\nE4UAAJBEIQAATBQCAEAShQAAMFEIAABJFAIAwEQhAAAkUQgAABOFAACQRCFEpbW1NdERRpmOmaTp\nmYtME0OmiZuuuSYrJoXQ0tKi559/Xjk5OXr33XfHHFNZWamcnBy5XC51dnbGIoblpuM//nTMJE3P\nXGSaGDJN3HTNNVmWF8LIyIjKy8vV0tKiixcv6o9//KP+8Y9/RIxpamrS5cuX1dXVpffff19lZWVW\nxwAARMnyQujo6FB2draee+45zZw5U8XFxfr4448jxvh8PpWUlEiSPB6PBgYG1NfXZ3UUAEA0DIv9\n6U9/Mnbu3Bl+/sEHHxjl5eURY9atW2ecO3cu/HzVqlXGhQsXIsZI4ocffvjhZxI/k5Usi9lstgmN\nMx66CfTDf+7h9QCA2LL8kJHD4ZDf7w8/9/v9yszMfOyYQCAgh8NhdRQAQBQsL4SlS5eqq6tLPT09\nGh4e1kcffaSioqKIMUVFRTp+/Lgkqb29XWlpacrIyLA6CgAgCpYfMkpOTtbRo0dVUFCgkZER7dix\nQ7m5uaqrq5MklZaWas2aNWpqalJ2drZmzZqlY8eOWR0DABCtSZ99sEhzc7Px4x//2MjOzjbeeeed\nMcdUVFQY2dnZxpIlS4yvvvoq4ZlOnTplfP/73zfy8vKMvLw847e//W3MM23fvt2YN2+esXjx4keO\nifc8jZcpEfN09epVw+v1Gk6n01i0aJFRU1Mz5rh4ztVEMsV7ru7evWv85Cc/MVwul5Gbm2v8+te/\nHnNcvN9TE8mViPeVYRjG/fv3jby8PGPdunVjro/3XI2XaTLzlNBCuH//vrFw4ULjypUrxvDwsOFy\nuYyLFy9GjGlsbDQKCwsNwzCM9vZ2w+PxJDzTqVOnjPXr18c0x8POnDljfPXVV4/c+MZ7niaSKRHz\ndP36daOzs9MwDMO4ffu28aMf/Sjh76mJZErEXN25c8cwDMMIBoOGx+Mx/vrXv0asT8R7aiK5EjFX\nhmEYv/vd74xXXnllzNdO1Fw9LtNk5imhX10xHa9ZmEgmKf6fglq+fLnmzp37yPWJuLZjvExS/Odp\n/vz5ysvLkyTNnj1bubm5unbtWsSYeM/VRDJJ8Z8ru90uSRoeHtbIyIjS09Mj1ifqeqHxcknxn6tA\nIKCmpibt3LlzzNdOxFyNl0mKfp4SWgi9vb3KysoKP8/MzFRvb++4YwKBQEIz2Ww2tbW1yeVyac2a\nNbp48WLM8kxUvOdpIhI9Tz09Pers7JTH44lYnsi5elSmRMxVKBRSXl6eMjIytHLlSjmdzoj1iZqn\n8XIlYq727NmjQ4cOacaMsTeZiZir8TJNZp4SWghWXbNgpYn83fn5+fL7/fr73/+uiooKbdiwIWZ5\nohHPeZqIRM7T4OCgNm3apJqaGs2ePXvU+kTM1eMyJWKuZsyYoa+//lqBQEBnzpwZ83t5EjFP4+WK\n91ydOHFC8+bNk9vtfuxv3PGcq4lkmsw8JbQQpuM1CxPJNGfOnPBubWFhoYLBoG7cuBGzTBMxHa/t\nSNQ8BYNBbdy4Ua+++uqY/wkSMVfjZUrkeyo1NVVr167VhQsXIpYn+j31qFzxnqu2tjb5fD4tWLBA\nW7Zs0eeff65t27ZFjIn3XE0k06TmafKnM6YuGAwaP/zhD40rV64Y9+7dG/ek8hdffBHzkzUTyfSv\nf/3LCIVChmEYxvnz541nn302ppm+c+XKlQmdVI7HPE0kUyLmKRQKGVu3bjV27979yDHxnquJZIr3\nXP373/82bt68aRiGYfznP/8xli9fbnz22WcRYxLxnppIrkT9/zMMw2htbR3zEz2J+v/3uEyTmSfL\nr0OIxnS8ZmEimRoaGlRbW6vk5GTZ7XbV19fHNJMkbdmyRadPn1Z/f7+ysrK0f/9+BYPBcKZEXNsx\nXqZEzNO5c+f04YcfasmSJXK73ZKkgwcP6urVq+Fc8Z6riWSK91xdv35dJSUlCoVCCoVC2rp1q1at\nWpXw64UmkisR76sHfXcoKNFzNV6mycyTzTD40iAAAHdMAwCYKAQAgCQKAQBgohAAAJIoBGDC/va3\nv8nlcunevXu6c+eOFi9ePC2uUgeswqeMgCi89dZbGhoa0t27d5WVlaVf/epXiY4EWIZCAKIQDAa1\ndOlSfe9739MXX3yR8K8HAazEISMgCv39/bpz544GBwd19+7dRMcBLMUeAhCFoqIivfLKK/rnP/+p\n69ev68iRI4mOBFgmoV9dATxJjh8/rmeeeUbFxcUKhUL66U9/qtbWVnm93kRHAyzBHgIAQBLnEAAA\nJgoBACCJQgAAmCgEAIAkCgEAYKIQAACSpP8DOY0gtieCYswAAAAASUVORK5CYII=\n"
      }
     ],
     "prompt_number": 34
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "mean_x = np.mean(x)\n",
      "mean_y = np.mean(y)\n",
      "mean_xy = np.mean(x*y)\n",
      "mean_xx = np.mean(x**2)\n",
      "display(Math(latex('\\\\bar{x} = %0.2f' % mean_x)))\n",
      "display(Math(latex('\\\\bar{y} = %0.2f' % mean_y)))\n",
      "display(Math(latex('\\\\bar{xy} = %0.2f' % mean_xy)))\n",
      "display(Math(latex('\\\\bar{x^2} = %0.2f' % mean_xx)))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "latex": [
        "$$\\bar{x} = 0.50$$"
       ],
       "output_type": "display_data",
       "text": [
        "<IPython.core.display.Math object at 0x104e34f50>"
       ]
      },
      {
       "latex": [
        "$$\\bar{y} = 0.25$$"
       ],
       "output_type": "display_data",
       "text": [
        "<IPython.core.display.Math object at 0x104de05d0>"
       ]
      },
      {
       "latex": [
        "$$\\bar{xy} = 5.25$$"
       ],
       "output_type": "display_data",
       "text": [
        "<IPython.core.display.Math object at 0x104e34f50>"
       ]
      },
      {
       "latex": [
        "$$\\bar{x^2} = 5.50$$"
       ],
       "output_type": "display_data",
       "text": [
        "<IPython.core.display.Math object at 0x105c85650>"
       ]
      }
     ],
     "prompt_number": 60
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "### Calculate the Slope\n",
      "$$\\begin{align*}\n",
      "m &= \\frac {\\bar{x}\\bar{y} - \\bar{xy}} {(\\bar{x})^2 - \\bar{x^2}}\\\\\\\\\n",
      "&= \\frac {0.50 * 0.25 - 5.25} {0.50^2 - 5.5}\n",
      "\\end{align*}\n",
      "$$"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "m = (np.mean(x) * np.mean(y) - np.mean(x*y)) / (np.mean(x)**2 - np.mean(x**2))\n",
      "display(Math(latex('m = %0.2f' % m)))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "latex": [
        "$$m = 0.98$$"
       ],
       "output_type": "display_data",
       "text": [
        "<IPython.core.display.Math object at 0x104e34090>"
       ]
      }
     ],
     "prompt_number": 61
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "###Calculate the Y-intercept\n"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "display(Math(latex('b = \\\\bar{y} - m\\\\bar{x}')))\n",
      "b = np.mean(y) - m * np.mean(x)\n",
      "display(Math(latex('b = %0.2f' % b)))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "latex": [
        "$$b = \\bar{y} - m\\bar{x}$$"
       ],
       "output_type": "display_data",
       "text": [
        "<IPython.core.display.Math object at 0x105c84810>"
       ]
      },
      {
       "latex": [
        "$$b = -0.24$$"
       ],
       "output_type": "display_data",
       "text": [
        "<IPython.core.display.Math object at 0x105c84810>"
       ]
      }
     ],
     "prompt_number": 74
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "##Regression Line Example \n",
      "https://www.khanacademy.org/math/probability/regression/regression-correlation/v/regression-line-example"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "x = np.array([1,2,4])\n",
      "y = np.array([2,1,3])\n",
      "m = (np.mean(x) * np.mean(y) - np.mean(x*y)) / (np.mean(x)**2 - np.mean(x**2))\n",
      "b = np.mean(y) - m * np.mean(x)\n",
      "# Lets make a nice chart!\n",
      "plt.figure()\n",
      "ax1 = plt.axes(frameon=True)\n",
      "ax1.get_xaxis().tick_bottom()\n",
      "ax1.get_yaxis().tick_left()\n",
      "plt.axis([0, 4.5, 0, 4.5])\n",
      "plt.plot(x, y, 'ro')\n",
      "t = np.arange(x[0] - x[-1])\n",
      "# y = b + m * x\n",
      "plt.plot([0, 10], [b, (b + m * 10)], 'g')\n",
      "#plt.plot(x, y, 'rx')\n",
      "plt.xlabel('x')\n",
      "plt.ylabel('y')\n",
      "plt.show()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEICAYAAABfz4NwAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAGoFJREFUeJzt3X9MVff9x/HXlet0VzspS8DkQlortOWuES61u5vxx+1c\nS8FKXTSNNlVSNSUkgvrNmq3VZpi1po2ujT8WYv/R2CWzmVlSWsC1TUVrKTItXZp0icgk3ksdHbFW\ncaLIPd8/dqVegcu9cM49F3g+EpJ7z/nIffdTOC/e53PPPQ7DMAwBACa8SXYXAABIDgQCAEASgQAA\nCCMQAACSCAQAQBiBAACQZGEg9PX1yev1aunSpQP2NTQ0aMaMGfJ6vfJ6vXrllVesKgMAECOnVd94\n165d8ng8unLlyqD7Fy1apJqaGqteHgAQJ0s6hGAwqLq6Oq1fv15DXffG9XAAkFws6RA2b96sHTt2\n6PLly4PudzgcamxsVF5entxut3bu3CmPxzNgDAAgfiP9g9v0QHj//feVnp4ur9erhoaGQccUFBQo\nEAjI5XKpvr5ey5Yt05kzZwaMo4sAgPiM5o9ph9mfZfTSSy/p7bffltPpVE9Pjy5fvqzly5fr4MGD\nQ/6bWbNm6fTp00pLS/u+MIeDQACAOI3m2Gl6INzu2LFj2rlzp957772I7Z2dnUpPT5fD4VBzc7Oe\nfvpptbe3RxZGIABA3EZz7LTsXUa33Gpf9u3bJ0kqKyvT4cOHVV1dLafTKZfLpUOHDlldBgBgGJZ2\nCKNBhwAA8RvNsZMrlQEAkggEAEAYgQAAkEQgAADCCAQAgCQCAQAQRiAAACQRCACAMAIBACCJQAAA\nhBEIAABJBAIAIIxAAABIIhAAAGEEAgBAEoEAAAizLBD6+vrk9Xq1dOnSQfdXVlYqJydHeXl5amlp\nsaoMAECMLAuEXbt2yePx9N9C83Z1dXU6e/asWltb9dZbb6m8vNyqMgAAMbLknsrBYFB1dXXasmWL\n3njjjQH7a2pqVFpaKkny+Xy6dOmSOjs7lZGRETGuqqqq/7Hf75ff77eiXACALAqEzZs3a8eOHbp8\n+fKg+zs6OpSVldX/PDMzU8FgMGogAACsZfopo/fff1/p6enyer1Rb/R8577BTi0BABLH9EBobGxU\nTU2NZs2apVWrVunjjz/WmjVrIsa43W4FAoH+58FgUG632+xSAABxMD0Qtm/frkAgoHPnzunQoUP6\nxS9+oYMHD0aMKSkp6d/W1NSk1NTUAaeLAACJZckawu1unQrat2+fJKmsrEzFxcWqq6tTdna2pk2b\npv3791tdBgBgGA4j2ol+GzkcjqhrEACAgUZz7ORKZQCAJAIBABBGIAAAJBEIAIAwAgEAIIlAAACE\nEQgAAEkEAgAgjEAAAEgiEAAAYQQCAEASgQAACCMQAACSCAQAQBiBAACQZEEg9PT0yOfzKT8/Xx6P\nRy+++OKAMQ0NDZoxY4a8Xq+8Xq9eeeUVs8sAAMTJ9DumTZ06VUePHpXL5dLNmzc1f/58nThxQvPn\nz48Yt2jRItXU1Jj98gCAEbLklJHL5ZIk3bhxQ319fUpLSxswhruhAUByseSeyqFQSAUFBWpra1N5\nebk8Hk/EfofDocbGRuXl5cntdmvnzp0DxkhSVVVV/2O/3y+/329FuQAAWXxP5e+++06FhYV67bXX\nIg7mV65cUUpKilwul+rr67Vx40adOXMmsjDuqQwAcUvaeyrPmDFDS5Ys0alTpyK233XXXf2nlYqK\nitTb26uLFy9aWQoAYBimB0JXV5cuXbokSbp27Zo+/PBDeb3eiDGdnZ39Cdbc3CzDMAZdZwAAJI7p\nawgXLlxQaWmpQqGQQqGQVq9ercWLF2vfvn2SpLKyMh0+fFjV1dVyOp1yuVw6dOiQ2WUAAOJk6RrC\naLCGAADxS9o1BADA2EEgAAAkEQgAgDACAQAgiUAAAIQRCAAASQQCACCMQAAASCIQAABhBAIAQBKB\nAAAIIxAAAJIIBABAGIEAAJBEIAAAwggEAIAkCwKhp6dHPp9P+fn58ng8evHFFwcdV1lZqZycHOXl\n5amlpcXsMgDAMsdra7W1sFBVfr+2FhbqeG2t3SWZwvRbaE6dOlVHjx6Vy+XSzZs3NX/+fJ04cULz\n58/vH1NXV6ezZ8+qtbVVJ0+eVHl5uZqamswuBQBMd7y2Vn/buFGvtrX1b9sSfrxwyRK7yjKFJaeM\nXC6XJOnGjRvq6+tTWlpaxP6amhqVlpZKknw+ny5duqTOzk4rSgEAU32we3dEGEjSq21t+nDPHpsq\nMo/pHYIkhUIhFRQUqK2tTeXl5fJ4PBH7Ozo6lJWV1f88MzNTwWBQGRkZEeOqqqr6H/v9fvn9fivK\nBYCYOa9fH3R7Sk9PgisxnyWBMGnSJH3xxRf67rvvVFhYqIaGhgEH8ztvAu1wOAZ8n9sDAQCSwc0p\nUwbd3jd1aoIrMZ+l7zKaMWOGlixZolOnTkVsd7vdCgQC/c+DwaDcbreVpQCAKR6vrNSW2bMjtr00\ne7Yeq6iwqSLzmN4hdHV1yel0KjU1VdeuXdOHH36o3/3udxFjSkpKtHfvXq1cuVJNTU1KTU0dcLoI\nAJLRrYXjl/fsUUpPj/qmTtUTFRVjfkFZsiAQLly4oNLSUoVCIYVCIa1evVqLFy/Wvn37JEllZWUq\nLi5WXV2dsrOzNW3aNO3fv9/sMgDAMguXLEmqALh8/bI++tdHqj9bP6rv4zDuPJmfJBwOx4B1BgDA\n/9Zgv/zmS9W31qv+bL1OXzitn2f+XEXZRfq/ef834mMngQAAY8DtXUB9a71+kPIDFeUUqSi7SI/e\n+6im/WCapNEdOwkEAEhCg3UB87LmqSj7fyFw/4/vH/TdmQQCAIwDsXYB0RAIADAGjbQLiIZAAIAx\nwowuIBoCAQCSlBVdQDQEAgAkEau7gGgIBACwUaK7gGgIBABIMDu7gGgIBACwWDJ1AdEQCABggWTt\nAqIhEADABGOlC4iGQACAERqLXUA0BAIAxGg8dAHREAgAEMV46wKiIRAA4DbjvQuIJqkCIRAIaM2a\nNfrmm2/kcDj0/PPPq7KyMmJMQ0ODnnrqKd13332SpOXLl2vr1q2RhREIAOIwkbqAaEZz7DT9FpqT\nJ0/Wm2++qfz8fHV3d+vhhx/WY489ptzc3IhxixYtUk1NjdkvD2CCiNYF/Prnvx7XXYBVTA+EmTNn\naubMmZKk6dOnKzc3V19//fWAQOCvfwDxGqoL+PW8X0+oLsAqpgfC7drb29XS0iKfzxex3eFwqLGx\nUXl5eXK73dq5c6c8Hs+Af19VVdX/2O/3y+/3W1kugCRDF5BYli0qd3d3y+/3a+vWrVq2bFnEvitX\nriglJUUul0v19fXauHGjzpw5E1kYawjAhMRawOgk1aKyJPX29urJJ59UUVGRNm3aNOz4WbNm6fTp\n00pLS/u+MAIBmBAm8juCrJBUi8qGYWjdunXyeDxDhkFnZ6fS09PlcDjU3NwswzAiwgDA+MZaQHIy\nvUM4ceKEFi5cqDlz5vSn+vbt23X+/HlJUllZmf74xz+qurpaTqdTLpdLb7zxhn72s59FFkaHAIwb\ndAGJk3SnjMxAIABjG2sB9iAQANiOLiA5EAgAbEEXkHwIBAAJQReQ/AgEAJYZqgsozi6W/14/XUCS\nIRAAmIYuYGyzJBB2796t1atX6+677x5VcSNFIACJQxcwflhyYVpnZ6ceeeQRFRQUaO3atSosLOSv\nAmCc4DOCMJiop4xCoZA++OADHThwQKdOndLTTz+tdevWafbs2dYXRocAmIouYGKw7KMrJk2apJkz\nZyojI0MpKSn69ttvtWLFCv3yl7/Ujh07RvSCABKDLgDxGrJD2LVrlw4ePKgf//jHWr9+vX71q19p\n8uTJCoVCysnJUVtbm7WF0SEAcaMLgCUdwsWLF/XXv/5V99xzT8T2SZMm6b333hvRiwEwF10AzMTb\nToExhi4A0XAdAjCOcV0A4kEgAOMMXQBGikAAxji6AJiFQADGILoAWCGpAiEQCGjNmjX65ptv5HA4\n9Pzzz6uysnLAuMrKStXX18vlcunAgQPyer2RhREIUR2vrdUHu3fLef26bk6ZoscrK7VwyRK7y0IU\ndAFIhKS6p/LkyZP15ptvKj8/X93d3Xr44Yf12GOPKTc3t39MXV2dzp49q9bWVp08eVLl5eVqamoy\nu5Rx63htrf62caNeve1akC3hx4RCchmqC3hh3gt0AUg6pgfCzJkzNXPmTEnS9OnTlZubq6+//joi\nEGpqalRaWipJ8vl8unTpkjo7O5WRkWF2OePSB7t3R4SBJL3a1qaX9+whEGzGdQEYy0wPhNu1t7er\npaVFPp8vYntHR4eysrL6n2dmZioYDA4IhKqqqv7Hfr9ffr/fynLHDOf164NuT+npSXAlkOgCMH5Y\nFgjd3d1asWKFdu3apenTpw/Yf+c5rsH+aro9EPC9m1OmDLq9b+rUBFcyMdEFYLyyJBB6e3u1fPly\nPfvss1q2bNmA/W63W4FAoP95MBiU2+22opRx6fHKSm1pa4s4bfTS7Nl6oqLCxqrGN7oATASmB4Jh\nGFq3bp08Ho82bdo06JiSkhLt3btXK1euVFNTk1JTU1k/iMOtdYKX9+xRSk+P+qZO1RMVFawfmIgu\nABOR6W87PXHihBYuXKg5c+b0/8Js375d58+flySVlZVJkjZs2KAjR45o2rRp2r9/vwoKCiIL422n\nSDCuC8B4kFTXIZiFQIDVuC4A4xGBAMSILgDjHYEADIEuABMNgQDchi4AExmBgAmNLgD4HoGACefO\nLmCKc0p/ANAFYCIjEDDu0QUAsSEQMC7RBQDxIxAwLtAFAKNHIGDMogsAzEUgYMygCwCsRSAgqdEF\nAIlDICCp0AUA9iEQYDu6ACA5EAhIOLoAIDkRCEgIugAg+SVdIKxdu1a1tbVKT0/Xl19+OWB/Q0OD\nnnrqKd13332SpOXLl2vr1q2RhREItqMLAMaepAuETz75RNOnT9eaNWuGDIQ33nhDNTU1QxdGINiC\nLgAY20Zz7DT9nsqStGDBArW3t0cdw8E+OXDvYAC3WBIIw3E4HGpsbFReXp7cbrd27twpj8czYFxV\nVVX/Y7/fL7/fn7gix7GhuoAX5r1AFwBMYJYtKre3t2vp0qWDnjK6cuWKUlJS5HK5VF9fr40bN+rM\nmTORhXHKyDSsBQATR9KtIUjRA+FOs2bN0unTp5WWlvZ9YQTCqLAWAExMSbeGMJzOzk6lp6fL4XCo\nublZhmFEhAHix1oAgNGyJBBWrVqlY8eOqaurS1lZWdq2bZt6e3slSWVlZTp8+LCqq6vldDrlcrl0\n6NAhK8oY91gLAGAmLkwbQ1gLADCcpFxDGC0C4X9YCwAQDwJhHKELADAaBMIYRxcAwCwEwhhDFwDA\nKgTCGEAXACARCIQkRBcAwA4EQpKgCwBgNwLBJtG6gOKcYuWk5dAFAEgoAiGBonUBj856VK7JLrtL\nBDCBEQgWGqoLKM4uVlFOEV0AgKRCIJiMLgDAWEUgjBJdAIDxgkAYAboAAOMRgRADugAAEwGBMAS6\nAAATDYEQRhcAYKJLqkBYu3atamtrlZ6ePuT9lCsrK1VfXy+Xy6UDBw7I6/UOLCzG/yi6AAzneG2t\nPti9W87r13VzyhQ9XlmphUuW2F0WYImkuqfyc889p4qKCq1Zs2bQ/XV1dTp79qxaW1t18uRJlZeX\nq6mpKebvH60LeGHeC3QBiHC8tlZ/27hRr7a19W/bEn5MKACRTA+EBQsWqL29fcj9NTU1Ki0tlST5\nfD5dunRJnZ2dysjIGPLfRLt3MF0Aovlg9+6IMJCkV9va9PKePQQCcAfTA2E4HR0dysrK6n+emZmp\nYDA4aCA8tvYxtV5s1YXuC5rjm6Nnlz5LF4C4OK9fH3R7Sk9PgisBkl/CA0HSgPNbQx3cH1jxgDZl\nb6ILwIjdnDJl0O19U6cmuBIg+SU8ENxutwKBQP/zYDAot9s96Ni9xXsTVRbGqccrK7WlrS3itNFL\ns2friYoKG6sCklPCA6GkpER79+7VypUr1dTUpNTU1KjrB8Bo3FoneHnPHqX09Khv6lQ9UVHB+gEw\nCNPfdrpq1SodO3ZMXV1dysjI0LZt29Tb2ytJKisrkyRt2LBBR44c0bRp07R//34VFBQMLCxJPu0U\nAMaSpLoOwSwEAgDEbzTHzkkm1wIAGKMIBACAJAIBABBGIAAAJBEIAIAwAgEAIIlAAACEEQgAAEkE\nAgAgjEAAAEgiEAAAYQQCAEASgQAACCMQAACSCAQAQBiBAACQZFEgHDlyRA8++KBycnL0+uuvD9jf\n0NCgGTNmyOv1yuv16pVXXrGiDABAHEy/p3JfX582bNigjz76SG63W4888ohKSkqUm5sbMW7RokWq\nqakx++UBACNkeofQ3Nys7Oxs3XvvvZo8ebJWrlypd999d8A4bo8JAMnF9A6ho6NDWVlZ/c8zMzN1\n8uTJiDEOh0ONjY3Ky8uT2+3Wzp075fF4Bnyvqqqq/sd+v19+v9/scgEAYaYHgsPhGHZMQUGBAoGA\nXC6X6uvrtWzZMp05c2bAuNsDAQBgLdNPGbndbgUCgf7ngUBAmZmZEWPuuusuuVwuSVJRUZF6e3t1\n8eJFs0sBAMTB9ECYO3euWltb1d7erhs3buidd95RSUlJxJjOzs7+NYTm5mYZhqG0tDSzSwEAxMH0\nU0ZOp1N79+5VYWGh+vr6tG7dOuXm5mrfvn2SpLKyMh0+fFjV1dVyOp1yuVw6dOiQ2WUAAOLkMJL0\n7T4Oh4N3IgFAnEZz7ORKZQCAJAIBABBGIAAAJBEIAIAwAgEAIIlAAACEEQgAAEkEAgAgjEAAAEgi\nEAAAYQQCAEASgQAACCMQAACSCAQAQBiBAACQRCDEpaGhwe4SBkjGmqTkrIuaYkNNsUvWukbKkkA4\ncuSIHnzwQeXk5Oj1118fdExlZaVycnKUl5enlpYWK8owXTL+z0/GmqTkrIuaYkNNsUvWukbK9EDo\n6+vThg0bdOTIEX311Vf685//rH/+858RY+rq6nT27Fm1trbqrbfeUnl5udllAADiZHogNDc3Kzs7\nW/fee68mT56slStX6t13340YU1NTo9LSUkmSz+fTpUuX1NnZaXYpAIB4GCb7y1/+Yqxfv77/+dtv\nv21s2LAhYsyTTz5pfPrpp/3PFy9ebJw6dSpijCS++OKLL75G8DVSTpnM4XDENM644ybQd/67O/cD\nAKxl+ikjt9utQCDQ/zwQCCgzMzPqmGAwKLfbbXYpAIA4mB4Ic+fOVWtrq9rb23Xjxg298847Kikp\niRhTUlKigwcPSpKampqUmpqqjIwMs0sBAMTB9FNGTqdTe/fuVWFhofr6+rRu3Trl5uZq3759kqSy\nsjIVFxerrq5O2dnZmjZtmvbv3292GQCAeI149cEk9fX1xgMPPGBkZ2cbr7322qBjKioqjOzsbGPO\nnDnG559/bntNR48eNX70ox8Z+fn5Rn5+vvH73//e8pqee+45Iz093XjooYeGHJPoeRquJjvm6fz5\n84bf7zc8Ho/xk5/8xNi1a9eg4xI5V7HUlOi5unbtmvHTn/7UyMvLM3Jzc43f/va3g45L9M9ULHXZ\n8XNlGIZx8+ZNIz8/33jyyScH3Z/ouRquppHMk62BcPPmTWP27NnGuXPnjBs3bhh5eXnGV199FTGm\ntrbWKCoqMgzDMJqamgyfz2d7TUePHjWWLl1qaR13On78uPH5558PefBN9DzFUpMd83ThwgWjpaXF\nMAzDuHLlinH//ffb/jMVS012zNXVq1cNwzCM3t5ew+fzGZ988knEfjt+pmKpy465MgzD+MMf/mA8\n88wzg762XXMVraaRzJOtH12RjNcsxFKTlPh3QS1YsEB33333kPvtuLZjuJqkxM/TzJkzlZ+fL0ma\nPn26cnNz9fXXX0eMSfRcxVKTlPi5crlckqQbN26or69PaWlpEfvtul5ouLqkxM9VMBhUXV2d1q9f\nP+hr2zFXw9UkxT9PtgZCR0eHsrKy+p9nZmaqo6Nj2DHBYNDWmhwOhxobG5WXl6fi4mJ99dVXltUT\nq0TPUyzsnqf29na1tLTI5/NFbLdzroaqyY65CoVCys/PV0ZGhh599FF5PJ6I/XbN03B12TFXmzdv\n1o4dOzRp0uCHTDvmariaRjJPtgaCWdcsmCmW711QUKBAIKB//OMfqqio0LJlyyyrJx6JnKdY2DlP\n3d3dWrFihXbt2qXp06cP2G/HXEWryY65mjRpkr744gsFg0EdP3580M/lsWOehqsr0XP1/vvvKz09\nXV6vN+pf3Imcq1hqGsk82RoIyXjNQiw13XXXXf1tbVFRkXp7e3Xx4kXLaopFMl7bYdc89fb2avny\n5Xr22WcH/SWwY66Gq8nOn6kZM2ZoyZIlOnXqVMR2u3+mhqor0XPV2NiompoazZo1S6tWrdLHH3+s\nNWvWRIxJ9FzFUtOI5mnkyxmj19vba9x3333GuXPnjOvXrw+7qPzZZ59ZvlgTS03//ve/jVAoZBiG\nYZw8edK45557LK3plnPnzsW0qJyIeYqlJjvmKRQKGatXrzY2bdo05JhEz1UsNSV6rv7zn/8Y3377\nrWEYhvHf//7XWLBggfHRRx9FjLHjZyqWuuz6/TMMw2hoaBj0HT12/f5Fq2kk82T6dQjxSMZrFmKp\n6fDhw6qurpbT6ZTL5dKhQ4csrUmSVq1apWPHjqmrq0tZWVnatm2bent7+2uy49qO4WqyY54+/fRT\n/elPf9KcOXPk9XolSdu3b9f58+f760r0XMVSU6Ln6sKFCyotLVUoFFIoFNLq1au1ePFi268XiqUu\nO36ubnfrVJDdczVcTSOZJ4dh8KFBAADumAYACCMQAACSCAQAQBiBAACQRCAAMfv73/+uvLw8Xb9+\nXVevXtVDDz2UFFepA2bhXUZAHF5++WX19PTo2rVrysrK0m9+8xu7SwJMQyAAcejt7dXcuXP1wx/+\nUJ999pntHw8CmIlTRkAcurq6dPXqVXV3d+vatWt2lwOYig4BiENJSYmeeeYZ/etf/9KFCxe0Z88e\nu0sCTGPrR1cAY8nBgwc1ZcoUrVy5UqFQSPPmzVNDQ4P8fr/dpQGmoEMAAEhiDQEAEEYgAAAkEQgA\ngDACAQAgiUAAAIQRCAAASdL/AwGcxEd1rWRhAAAAAElFTkSuQmCC\n"
      }
     ],
     "prompt_number": 56
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Second Regression Example \n",
      "https://www.khanacademy.org/math/probability/regression/regression-correlation/v/second-regression-example"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "x = np.array([-2, -1, 1, 4])\n",
      "y = np.array([-3, -1, 2, 3])\n",
      "m = (np.mean(x) * np.mean(y) - np.mean(x*y)) / (np.mean(x)**2 - np.mean(x**2))\n",
      "b = np.mean(y) - m * np.mean(x)\n",
      "plt.figure()\n",
      "ax1 = plt.axes(frameon=True)\n",
      "ax1.get_xaxis().tick_bottom()\n",
      "ax1.get_yaxis().tick_left()\n",
      "plt.axis([-4.5, 4.5, -4.5, 4.5])\n",
      "plt.plot(x, y, 'ro')\n",
      "t = np.arange(x[0] - x[-1])\n",
      "# y = b + m * x\n",
      "plt.plot([-10, 0, 10], [(b + m * -10), b, (b + m * 10)], 'g')\n",
      "plt.xlabel('x')\n",
      "plt.ylabel('y')\n",
      "plt.title('m = %f, b = %f' % (m, b))\n",
      "plt.show()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAXsAAAETCAYAAADXmaY8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XtcVHX+P/AXNyUcMUHFELwiCg4gCmquKf2UMG/rfSXz\nBqhAael3Xbvo5rpprm72EE3QkjUqqXa1ZOSiRIiEkWiIjJoWaYiKoiA6KJeZ+fz+aJsVAUEc5sww\nr+fj4ePBnHPgvBh8vHx7zuEcCyGEABERtWqWUgcgIqKWx7InIjIDLHsiIjPAsiciMgMseyIiM8Cy\nJyIyAyx7osc0f/58rF69WuoYRA/FsqfHtmfPHvTo0QMymQxTpkxBWVlZg9sePXoUQ4YMgb29PXx8\nfJCVlaVbt379erRv3173x87ODlZWVigtLdVt8/XXX2PQoEGQyWRwdXXFv//9b926RYsWoX///rCy\nssJHH31Ua79VVVVYtmwZunXrBgcHB7z00ktQq9W69aWlpZgyZQpkMhl69uyJ+Pj4Jn//FhYWsLCw\naPL2+vIomT/66CP4+fmhQ4cOcHV1xcqVK6HRaHTrX3zxRTz11FOwt7dH7969sW7dulqf/9VXX2HA\ngAGwt7fHgAEDsH///lrrV65ciU6dOqFTp0547bXXaq172M/88OHDsLS0rPVz//jjjx/nbaGGCKLH\noFQqRfv27UVmZqZQqVTihRdeELNmzap325s3bwoHBwfxn//8R2i1WvHJJ5+Ijh07irKysnq3X7Nm\njRg9erTu9enTp0WXLl1ESkqK0Gg0orS0VBQUFOjWv//++yItLU34+fmJjz76qM7XGjlypCgrKxMl\nJSVi2LBh4q233tKtnzVrlpg1a5aoqKgQ3377rejQoYM4ffp0k96D+fPni1WrVjVpW316lMzR0dHi\n22+/FTU1NeLy5cti8ODBYsOGDbr1SqVS3Lt3TwghxI8//iicnJxEcnKyEEKIa9euCTs7O5GSkiKE\nECIxMVHY2dmJkpISIYQQMTExol+/fuLy5cvi8uXLwtPTU8TExAghGv+Zp6enCxcXl5Z5g6gWln0r\n0qNHD7Fp0ybh5eUlZDKZCAkJEcXFxWLs2LHC3t5ejBkzpsFiba7XX39dzJ49W/e6oKBAtGnTRqhU\nqjrbKhQK4enpWWuZu7u72LVrV51ttVqt6NWrl4iLi9MtCw4OFn/9618bzTRixIg6Ze/n5yf+/e9/\n617v2bNHuLq6CiGEUKlUok2bNuKnn37SrZ87d6547bXXGt2XEL+VfXh4uAgMDBTt27cXo0aNEr/+\n+muTPre5Hjfz5s2bxcSJE+td9+OPP4pu3bqJEydOCCGEyMrKEl26dKm1TefOnUV2drYQQoinn35a\nfPDBB7p1sbGxYtiwYUKIxn/mLHvD4WGcVsTCwgL79u1DWloazp07hwMHDuD555/Hhg0bcP36dWi1\nWkRFRdX7uYWFhejYsWODfz777LN6P+/MmTPw8fHRve7duzfatm2L8+fPNymzVqvF6dOn6yzPzMxE\nSUkJpk2bplv2/fffQwgBb29vODs7Y86cOQ89ZPQgcd+dQbRaLYqKinDnzh2cP38e1tbWcHNz0633\n8fGpN1dDX/fTTz/FX//6V9y4cQMDBw7E7NmzG9z+ySefbPB93rhxY5P2+biZMzIyIJfLay2LjIxE\nu3btMGDAAKxatQqDBg3SfV1ra2scOHAAGo0GX331FWxtbeHt7Q2g7t8Bb2/vh+Z48Gd+/fp1dO3a\nFb1798by5ctx9+7dJn0P9Igk/seG9Khnz55iz549utfTpk0TkZGRutdbt24VkydP1us+R48eLXbs\n2FFrWbdu3URGRkadbW/cuCE6duwoPvvsM1FdXS12794tLC0tRXh4eJ1tQ0JCxIIFC2ots7GxEb16\n9RI//fSTUKlUYtq0abX+V/G7+ib7VatWiT/84Q+ipKREXL16VQwZMkRYWlqK4uJiceTIEdG1a9da\n2+/cuVMEBAQ06T2YN2+eCA4O1r1WqVTCyspKFBUVNenzm+NxMu/atUu4urqKmzdv1lmn1WpFenq6\ncHR0FN9//71uuUKhEHZ2dsLa2lrY2dmJxMRE3TorKytx7tw53evz588LCwsLIUTjP/Pi4mJx9uxZ\nIYQQFy5cECNHjhSLFy9+hHeCmoqTfSvj5OSk+/iJJ56o9drW1hYqlUqv+5PJZCgvL6+1rLy8HO3b\nt6+zraOjI7766iu8++676Nq1Kw4ePIgxY8bAxcWl1nZ3797Ff/7zH8ybN6/Wcjs7OyxYsABubm5o\n164d3njjDSQlJTUp55tvvglfX18MHDgQI0aMwJQpU2BtbQ0nJyfIZDLcvn27Sd9DfSwsLGp9D+3a\ntYODgwOuXLnSpM9viueff153AjM+Pr7Zmb/66iu88cYbSE5OhoODQ531FhYWCAgIwIwZM3QnfH/4\n4QcsWrQImZmZqKmpQUZGBsLCwnDq1CkAqJOlvLwcMpkMQOM/cycnJ/Tv3x8A0LNnT2zcuBF79+5t\n5rtED8Oyb+VEE29qWlhYWOuKiAf/NHSlx4ABA5CXl6d7XVBQgOrqari7u9e7/ciRI3Hs2DHcvHkT\ncXFx+PHHHzFkyJBa23z55ZdwdHTEqFGjai3//bBBc9ja2mLr1q0oKirCzz//DAcHB/j5+QEA3N3d\noVar8fPPP+u2z8vLq3OY42EuXbqk+1ilUqG0tBTOzs71biuTyRp8nzds2FDv5yQnJ+POnTu4c+cO\ngoODm5U5JSUFixYtwoEDBzBgwICHfj81NTVo164dACAtLQ3Dhg3THdbx8/PD0KFD8fXXXwP47e/A\nyZMnG8zRlJ/5/bRa7UOzUTNJ/V8L0p+ePXuKtLQ03esXX3xRrFmzRvf6gw8+EGPGjNHrPk+fPi3s\n7e11V+MEBwfXOqTxoB9++EFUV1eL8vJy8corr4gRI0bU2SYwMLDWlTK/i42NFb169RK//PKLqKio\nEDNmzBBz587Vra+urhb37t0Tw4cPFx988IG4d++e0Gq1Qgihu1JEq9WK7777Tri6uorU1FTd586a\nNUsEBweLiooKkZmZKTp06CDOnDmjW29hYVHvoSkhfjuMY29vL7799ltRVVUlXn311Xq/L31rLPP9\n0tLShIODg8jMzKyz7vr16yI+Pl6oVCqhVqtFSkqKsLe3F8eOHRNCCHHw4EHRqVMncfLkSSHEbz9D\nR0dH3fsXExMjPDw8xOXLl0VRUZHw9PSsdWjvYT/z9PR0cfHiRaHVakVhYaEYNWqUCAkJ0dt7RP/D\nsm9F6iv7v/3tb7rXH374oQgMDNT7fvfs2SO6d+8u2rVrJyZPnlzrip/w8PBax+SDg4NFhw4dRIcO\nHcSsWbN0l+/9rqioSNjY2NS6pPJ+b731lujcubPo3LmzmDt3rrh165Zu3ahRo4SFhYWwtLQUFhYW\ntQr6yJEjomfPnsLOzk7079+/1rkNIYQoLS0VkydPFu3atRM9evQQ8fHxunWFhYXC3t5elJaW1ptp\n/vz5IiIiQgQGBgqZTCZGjRolLl682MR3r/kelvnXX38VMplMXLp0SQghxLPPPitsbGyETCbT/Rk3\nbpwQQoiSkhIxatQo8eSTT4oOHToIf39/sX///lr72rhxo+jdu7eQyWSid+/eYvPmzbXW/+UvfxEO\nDg7CwcFBrFy5sta6h/3MN2/eLLp16ybs7OyEq6ureOWVV+q9kosen4UQfHgJ0cN8+umnOHPmTJ1f\nNCIyJSx7IiIzwBO0RERmgGVPRGQGrKUO8CApbihFRNQaPOyovNGVPdD0a8OJiOg3jQ3KPIxDRGQG\nWPZERGaAZU9EZAZY9kREZoBlT0RkBlj2RERmgGVPRGQGJCl7jUYDX19fTJw4UYrdExG1GmqtGhuz\nGn+cpSS/VLVlyxZ4enrizp07UuyeiKhVyCvOQ0hCCDradmx0W4NP9kVFRUhKSkJYWBh/U5aIqBmq\n1FVYnb4agR8H4iX/l5A6J7XRzzH4ZL9s2TJs2rSpzvMz77dmzRrdxwEBAQgICGj5YEREJiC7KBsh\n+0Pg7uiOk+En4dy+/sdfPsigZX/gwAF06dIFvr6+OHz4cIPb3V/2REQEVFRXYFX6Knym/Axbxm7B\nDM8Zj3TjSIMexjl69CgSEhLQq1cvBAcH45tvvsHcuXMNGYGIyOSk/ZIGr2gv3Lh7A/kR+Zg5YOYj\n3yFYsidVZWRk4J///CcUCkXtQBYWPJZPRATgVuUtrEhdgYM/H0T0+GiMdx/f4LaNdaek19nz3vVE\nRPVLOJcA+XY5rC2toYxUPrTom8LonkHLyZ6IzFlJRQmWJC/B8SvH8eGkDxHQM6BJn2fUkz0REf1G\nCIE9+XvgFe0F1w6uOBVxqslF3xRG+aQqIiJzUnS7COEHwvFr+a9QBCvg381f7/vgZE9EJBGt0GLn\niZ3w3eELf2d/nFh0okWKHuBkT0QkiZ9Lf8ZCxULcrbmL9HnpkHeRt+j+ONkTERmQRqvBu0ffxbAP\nh2FC3wk4GnK0xYse4GRPRGQwyutKhCaEws7GDtlh2XBzcDPYvjnZExG1sGpNNf52+G949qNnEeob\nirS5aQYteoCTPRFRi8q5nIOQhBD06NADuYtz4WLvIkkOlj0RUQu4W3MXbx1+Cx/nfYzNQZsRLA+W\n9K4BLHsiIj3LuJiBMEUY/Jz9cCriFLq06yJ1JJY9EZG+3K66jZVfr4TinALbx2/HpH6TpI6kwxO0\nRER6kPRTEuTb5VBr1VBGKo2q6AFO9kREj+XG3RtYdnAZsgqz8K8//guje4+WOlK9ONkTETWDEAJf\nnP4CXtFe6GTXCfkR+UZb9AAneyKiR3blzhW8lPQSzt04h30z9+Fp16eljtQoTvZERE0khEBsbiwG\nxgyEvIscuYtzTaLoAU72RERNcqHsAhYdWISye2VInZMKn64+zf5aRxITcSgqCtZVVVC3bYvnli7F\nyPGP9ySqxrDsiYgeQqPV4P2c97E2Yy3+8oe/YPnTy2Ft2fzqPJKYiIOvvIJ1BQW6ZW/+9+OWLHw+\nlpCIqAFnS84iNCEUVpZW2DVpF9wd3R/7a64KCsLbhw7VWb46KAh/T0lp9tflYwmJiB5RjaYG646s\nwzP/egYver+IjPkZeil6ALCuqqp3uVVlpV6+foP7bdGvTkRkYn64+gNCE0Lh1M4JJxadQI8ne+j1\n66vbtq13ucbWVq/7eRAneyIiAJXqSrye9jrGfjIWy4YtQ/LsZL0XPQA8t3Qp3uzTp9ayN/r0QeCS\nJXrf1/042ROR2fu28FuEJoTC28kbpyJOoausa4vt6/eTsKu3boVVZSU0trYYu2RJi1+NwxO0RGS2\n7lTdwRvfvIG9Z/Zi27htmOoxVepIzcYTtERE9ThUcAhe0V5QVaugjFSadNE3BQ/jEJFZKbtXhuWH\nliP9Qjp2TNiBILcgqSMZBCd7IjIb+87ugzxaDlkbGfIj8s2m6AFO9kRkBopVxXg56WUoryvx+fTP\nMaL7CKkjGRwneyJqtYQQiMuLg0+MD/o69sXJ8JNmWfQAJ3siaqUKywux+MBiXL1zFcmzkzHoqUFS\nR5IUJ3sialW0QovtOdsxeOdgPNP9GeQszDH7ogc42RNRK3L+5nmEJYRBrVXjyPwj8OjsIXUko8HJ\nnohMnlqrxsasjRi+azime05H5oJMFv0DONkTkUnLK85DaEIoOj7RETkLc9CrYy+pIxkllj0RmaQq\ndRXeznwbO47vwIYxG7Bg4AJYWFhIHctoseyJyORkF2UjZH8I3B3dcTL8JJzbO0sdyeix7InIZFRU\nV2BV+ip8pvwMW8ZuwQzPGZzmm4gnaInIJKT9kgavaC/cuHsD+RH5mDlgJov+EXCyJyKjdqvyFlak\nrsDBnw8ienw0xru37H3fWytO9kRktBLOJUC+XQ5rS2soI5Us+sfAyZ6IjE5JRQmWpizF8SvH8enU\nTzGq5yipI5k8g0/2ly5dwrPPPosBAwZALpcjKirK0BGIWoUjiYlYFRSENQEBWBUUhCOJiVJHemxC\nCOzJ3wOvaC+42LsgLzyPRa8nBp/sbWxs8N5772HgwIFQqVQYPHgwAgMD4eHB33YjaqojiYk4+Mor\nWFdQoFv25n8/bulnmbaUottFiEiMwMVbF6EIVsC/m7/UkVoVg0/2Xbt2xcCBAwEAMpkMHh4euHLl\niqFjEJm0Q1FRtYoeANYVFCB161aJEjWfVmix88RO+O7whb+zP04sOsGibwGSHrO/ePEicnNzMXTo\n0FrL16xZo/s4ICAAAQEBhg1GZOSsq6rqXW5VWWngJI+noLQAYYow3K25i/R56ZB3kUsdqdWSrOxV\nKhWmT5+OLVu2QCaT1Vp3f9kTUV3qtm3rXa6xtTVwkubRaDXY8v0WrM9cj9dHvI5Xh70KK0srqWO1\napKUfU1NDaZNm4YXX3wRkydPliICkUl7bulSvFlQUOtQzht9+mDskiUSpmoa5XUlQhNCYWdjh+yw\nbLg5uEkdySxYCCGEIXcohMC8efPg6OiI9957r24gCwsYOBKRSTqSmIjUrVthVVkJja0tApcsMeqT\ns9WaaryT+Q625WzDuv+3DmGDwmBpwV/10ZfGutPgZf/tt99i5MiR8Pb21v2q8zvvvIOxY8f+Fohl\nT9Tq5FzOQWhCKLp36I6YCTFwsXeROlKrY3Rl3xiWPVHrcbfmLtYcXoO4vDhsDtqMYHkw72fTQhrr\nTv4GLRG1iIyLGQhThMHP2Q+nIk6hS7suUkcyayx7ItKr21W3sfLrlVCcU2D7+O2Y1G+S1JEIvBEa\nEelR0k9J8Ir2glqrhjJSyaI3Ipzsieix3bx7E68efBVZhVmInRSL0b1HSx2JHsDJnoiaTQiBL05/\nAXm0HJ3sOiE/Ip9Fb6Q42RNRs1y5cwUvJb2EczfO4cs/fYlhLsOkjkQPwcmeiB6JEAKxubEYGDMQ\n8i5y5C7OZdGbAE72RNRkF8ouYNGBRSi7V4bUOanw6eojdSRqIk72RNQojVaDqO+j4P+BPwJ7ByI7\nLJtFb2I42RPRQ50tOYswRRisLKxwNPQo3B3dpY5EzcDJnojqVaOpwfrM9Ri5eyRme83G4fmHWfQm\njJM9EdWRezUXIQkhcGrnhOMLj6PHkz2kjkSPiWVPRDqV6kqszViLXbm7sClwE+Z4z+GNy1oJlj0R\nAQCyCrMQmhAKLycvnAo/BSeZk9SRSI9Y9kRmTlWtwutpr2Pvmb3YNm4bpnpMlToStQCeoCUyY4cK\nDsEr2guqahWUkUoWfSvGyZ7IDJXdK8PyQ8uRfiEdOybsQJBbkNSRqIVxsicyM/vO7oM8Wg5ZGxny\nI/JZ9GaCkz2RmbimuoaXk19G/rV8fD79c4zoPkLqSGRAnOyJWjkhBOLy4uAd4w03BzecDD/JojdD\nnOyJWrHC8kIsPrAYxapiJM9OxqCnBkkdiSTCyZ6oFdIKLbbnbMfgnYPxTPdncCzsGIvezHGyJ2pl\nzt88j7CEMKi1ahyZfwQenT2kjkRGgJM9USuh1qqxMWsjhu8ajume05G5IJNFTzqc7IlagbziPIQm\nhKLjEx2RszAHvTr2kjoSGRmWPZEJq1JX4e3Mt7Hj+A5sGLMBCwYu4I3LqF4seyITlV2UjdCEULg7\nuuNk+Ek4t3eWOhIZMZY9kYmpqK7A6vTViFfGY8vYLZjhOYPTPDWKJ2iJTMg3F76Bd4w3Su6WQBmh\nxMwBM1n01CSc7IlMQHllOf6c+mcc/PkgYibEYFzfcVJHIhPDyZ7IyCnOKSCPlsPa0hrKSCWLnpqF\nkz2RkSqpKMHSlKU4fuU4PpnyCUb1HCV1JDJhnOyJjIwQAvH58fCK9oKLvQvywvNY9PTYONkTGZGi\n20WISIzAr7d+hSJYAf9u/lJHolaCkz2RERBCYOeJnfDd4Qt/Z38cX3ScRU96xcmeSGIFpQVYqFiI\nipoKpM9Lh7yLXOpI1ApxsieSiEarwebvNmPoh0MxwX0CjoYcZdFTi+FkTySB09dPIyQhBHY2dsgO\ny4abg5vUkaiV42RPZEDVmmqszViLgI8CEOobirS5aSx6MghO9kQGknM5B6EJoejeoTtyF+fCxd5F\n6khkRiSZ7FNSUtC/f3/07dsX//jHP6SIQGQw92ru4S+pf8HE+Il4bcRrUAQrWPRkcBZCCGHIHWo0\nGvTr1w9ff/01unXrBn9/f8THx8PD47cn6lhYWMDAkYhaTMbFDIQpwuDn7IeosVHo3K6z1JGolWqs\nOxuc7KOiolBWVqb3QMeOHYObmxt69uwJGxsbzJo1C/v379f7foikdLvqNiISIzB732y8+9y7iJ8W\nz6InSTV4zP7atWvw9/fHoEGDEBISgqCgIL3cSvXy5ctwdXXVvXZxccH3339fa5s1a9boPg4ICEBA\nQMBj75fIUJJ/SkZ4Yjie6/MclJFKPGn7pNSRiBou+3Xr1uHvf/87Dh06hN27d+Pll1/GzJkzERoa\nij59+jR7h035B+P+sicyFTfv3sSyg8uQdSkLsZNiMbr3aKkjEek89AStpaUlunbtCicnJ1hZWaGs\nrAzTp0/HihUrmr3Dbt264dKlS7rXly5dgosLT1aR6RJC4IvTX0AeLYejnSNOhZ9i0ZPRafAE7ZYt\nWxAXFwdHR0eEhYVhypQpsLGxgVarRd++fVFQUNCsHarVavTr1w9paWlwdnbGkCFDeIKWTNbVO1cR\nmRSJczfOIfaPsRjmMkzqSGSmGuvOBg/jlJaWYt++fejRo0et5ZaWllAoFM0OZG1tjW3btiEoKAga\njQahoaG6oicyFUII7D65Gyu/Xolwv3B8Nu0ztLVuK3UsogYZ/NLLxnCyJ2N38dZFLFIsQum9Uuya\ntAs+XX2kjkTU/Esviag2rdAi6vso+H/gjzG9xyA7LJtFTyaDt0sgaoKzJWcRpgiDpYUlskKy4O7o\nLnUkokfCyZ7oIWo0NVifuR4jd4/EbK/ZyJifwaInk8TJnqgBuVdzEZIQgq6yrji+8Dh6PNmj8U8i\nMlIse6IHVKorsTZjLXbl7sKmwE2Y4z1HL789TiQllj3RfbIKsxCaEAovJy+cCj8FJ5mT1JGI9IJl\nTwRAVa3CG2lvYO/Zvdj6/FZM9ZgqdSQiveIJWjJ7hwoOwSvaC3eq70AZoWTRU6vEyZ7MVtm9Miw/\ntBzpF9KxY8IOBLkFSR2JqMVwsiez9OXZLyGPlkPWRob8iHwWPbV6nOzJrFxTXcPLyS8j/1o+Pp/+\nOUZ0HyF1JCKD4GRPZkEIgbi8OHjHeKOvQ1+cDD/JoiezwsmeWr3C8kIsPrAYxapiJM9OxqCnBkkd\nicjgONlTq6UVWmzP2Y7BOwfjme7P4FjYMRY9mS1O9tQqnb95HmEJYVBr1Tgy/wg8OvOZCWTeONlT\nq6LWqrExayOG7xqO6Z7Tkbkgk0VPBE72ZGSOJCbiUFQUrKuqoG7bFs8tXYqR48c36XNPXTuFkP0h\n6PhER+QszEGvjr1aOC2R6WDZk9E4kpiIg6+8gnX3Pd/4zf9+/LDCr1JXYV3mOsQcj8GGMRuwYOAC\n3riM6AE8jENG41BUVK2iB4B1BQVI3bq1wc/JLsrGoJ2DkH89HyfDTyLEN4RFT1QPTvZkNKyrqupd\nblVZWWdZRXUFVqevRrwyHlvGbsEMzxkseaKH4GRPRkPdtm29yzW2trVef3PhG3jHeKPkbgmUEUrM\nHDCTRU/UCE72ZDSeW7oUbxYU1DqU80afPhi7ZAkAoLyyHH9O/TMO/nwQMRNiMK7vOKmiEpkclj0Z\njd9Pwq7euhVWlZXQ2Npi7JIlGDl+PBTnFIhMisRE94lQRiph39Ze4rREpsVCCCGkDnE/CwsLGFkk\nklBJRQmWpizF8SvH8eHEDzGq5yipIxEZpca6k8fsySgJIRCfHw+vaC+42LsgLzyPRU/0GHgYh4xO\n0e0iRCRG4Ndbv0IRrIB/N3+pIxGZPE72ZDSEENh5Yid8d/jC39kfxxcdZ9ET6QknezIKBaUFWKhY\niIqaCqTPS4e8i1zqSEStCid7kpRGq8Hm7zZj6IdDMcF9Ao6GHGXRE7UATvYkmdPXTyMkIQR2NnbI\nDsuGm4Ob1JGIWi1O9mRw1ZpqrM1Yi4CPAhDqG4q0uWkseqIWxsmeDCrncg5CE0LRvUN35C7OhYu9\ni9SRiMwCy54M4l7NPbx1+C3E5cVhc9BmBMuDeT8bIgNi2VOLy7iYgTBFGPyc/ZAfkY/O7TpLHYnI\n7LDsqcXcrrqNlV+vhOKcAtvHb8ekfpOkjkRktniCllpE8k/J8Ir2glqrhjJSyaInkhgne9Krm3dv\nYtnBZci6lIXYSbEY3Xu01JGICJzsSU+EEPji9BeQR8vhaOeIU+GnWPRERoSTPT22q3euIjIpEudu\nnMOXf/oSw1yGSR2JiB7AyZ6aTQiB2NxY+MT4wKuLF3IX57LoiYwUJ3tqlou3LmKRYhFK75UidU4q\nfLr6SB2JiB7CoJP9ihUr4OHhAR8fH0ydOhXl5eWG3D3pgUarQdT3UfDb6YcxvccgOyybRU9kAgz6\nWMLU1FSMHj0alpaWeO211wAAGzZsqB2IjyU0WmdLziJMEQYrCyt8OOlDuDu6Sx2JiP7LqB5LGBgY\nCEvL33Y5dOhQFBUVGXL31Ew1mhqsz1yPkbtHYrbXbByef5hFT2RiJDtmHxsbi+Dg4HrXrVmzRvdx\nQEAAAgICDBOK6si9mouQhBA4tXPC8YXH0ePJHlJHIqJm0PthnMDAQBQXF9dZvn79ekycOBEAsG7d\nOvzwww/Yu3dv3UA8jGMUKtWVWJuxFrtyd2FT4CbM8Z7DG5cRGbHGulPvk31qaupD1+/evRtJSUlI\nS0vT965JT7IKsxCaEAovJy+cCj8FJ5mT1JGI6DEZ9DBOSkoKNm3ahIyMDNja2hpy19QEqmoVXk97\nHXvP7MW2cdsw1WOq1JGISE8MejVO3759UV1dDQcHBwDA008/je3bt9cOxMM4kjhUcAiLDyxGQM8A\nbH5uMzo+0VHqSET0CBrrToOWfVOw7A2r7F4Zlh9ajvQL6dgxYQeC3IKkjkREzWBUl16Scdl3dh/k\n0XLI2sh3UOKSAAAKwklEQVSQH5HPoidqxXi7BDN0TXUNLye/jPxr+fh8+ucY0X2E1JGIqIVxsjcj\nQgjE5cXBO8Ybbg5uOBl+kkVPZCY42ZuJwvJCLD6wGMWqYiTPTsagpwZJHYmIDIiTfSunFVpsz9mO\nwTsH45nuz+BY2DEWPZEZ4mTfip2/eR5hCWFQa9U4Mv8IPDp7SB2JiCTCyb4VUmvV2Ji1EcN3Dcd0\nz+nIXJDJoicyc5zsW5m84jyEJoSi4xMdkbMwB7069pI6EhEZAZZ9K1GlrsLbmW9jx/Ed2DBmAxYM\nXMAblxGRDsu+FcguykZoQijcHd1xMvwknNs7Sx2JiIwMy96EVVRXYHX6asQr47Fl7BbM8JzBaZ6I\n6sUTtCbqmwvfwDvGGyV3S6CMUGLmgJkseiJqECd7E1NeWY4/p/4ZB38+iJgJMRjXd5zUkYjIBHCy\nNyGKcwrIo+WwtrSGMlLJoieiJuNkbwJKKkqwNGUpjl85jk+mfIJRPUdJHYmITAwneyMmhMCe/D3w\nivaCi70L8sLzWPRE1Cyc7I1U0e0iRCRG4OKti1AEK+DfzV/v+ziSmIhDUVGwrqqCum1bPLd0KUaO\nH6/3/RCR9Fj2RkYIgQ9++ABvfvMmlgxZgr0z96KNVRu97+dIYiIOvvIK1hUU6Ja9+d+PWfhErQ8f\nS2hECkoLsFCxEBU1Fdg1aRfkXeQttq9VQUF4+9ChOstXBwXh7ykpLbZfImoZfCyhCdBoNdj83WYM\n/XAoJrhPwNGQoy1a9ABgXVVV73KrysoW3S8RSYOHcSR2+vpphCSEwM7GDtlh2XBzcDPIftVt29a7\nXGNra5D9E5FhcbKXSLWmGmsz1iLgowCE+oYibW6awYoeAJ5buhRv9ulTa9kbffogcMkSg2UgIsPh\nZC+BnMs5CE0IRfcO3ZG7OBcu9i4Gz/D7SdjVW7fCqrISGltbjF2yhCdniVopnqA1oLs1d7Hm8BrE\n5cVhc9BmBMuDeT8bItKLxrqTk72BZFzMQJgiDH7OfjgVcQpd2nWROhIRmRGWfQu7XXUbK79eCcU5\nBbaP345J/SZJHYmIzBBP0LagpJ+S4BXtBbVWDWWkkkVPRJLhZN8Cbt69iVcPvoqswizETorF6N6j\npY5ERGaOk70eCSHwxekvII+Wo5NdJ+RH5LPoicgocLLXk6t3riIyKRLnbpzDl3/6EsNchkkdiYhI\nh5P9YxJCIDY3Fj4xPpB3kSN3cS6LnoiMDif7x3Ch7AIWHViEsntlSJ2TCp+uPlJHIiKqFyf7ZtBo\nNYj6Pgr+H/gjsHcgssOyWfREZNQ42T+isyVnEaYIg5WFFY6GHoW7o7vUkYiIGsXJvolqNDVYn7ke\nI3ePxGyv2Tg8/zCLnohMBif7Jsi9mouQhBA4tXPC8YXH0ePJHlJHIiJ6JCz7h6hUV2Jtxlrsyt2F\nTYGbMMd7Dm9cRkQmiWXfgKzCLIQmhMLLyQunwk/BSeYkdSQiomZj2T9AVa3C62mvY++Zvdg2bhum\nekyVOhIR0WPjCdr7HCo4BPl2OVTVKigjlSx6Imo1JCn7d999F5aWligtLZVi93WU3SvDgv0LsEix\nCDsm7MC//vgvODzhIHUsIiK9MXjZX7p0CampqejRwziuaNl3dh/k0XLI2siQH5GPILcgqSMREemd\nwY/ZL1++HBs3bsQf//hHQ++6lmJVMZYkL0H+tXx8Pv1zjOg+QtI8REQtyaBlv3//fri4uMDb2/uh\n261Zs0b3cUBAAAICAvSWQQiBj099jBWpKxDiG4KPp3wMW2tbvX19IiJjpPeyDwwMRHFxcZ3l69at\nwzvvvINDhw7pljX0cNz7y16fCssLsfjAYhSripE8OxmDnhrUIvshIjI2FuJhjyPXI6VSidGjR8PO\nzg4AUFRUhG7duuHYsWPo0uV/D99u7AnpzaEVWsQcj8Fbh9/CsmHLsGL4CthY2eh1H0REUmqsOw1W\n9g/q1asXTpw4AQeH2le96Lvsz988j7CEMKi1auyatAsenT309rWJiIxFY90p2XX2LX3bAbVWjY1Z\nGzF813BM95yOzAWZLHoiMluSTfYN0cdkn1ech9CEUHR8oiN2TtiJXh176SkdEZFxaqw7W9XtEqrU\nVXg7823sOL4DG8ZswIKBC3jjMiIitKKyzy7KRmhCKPo69MXJ8JNwbu8sdSQiIqNh8mVfUV2BVemr\n8JnyM2wZuwUzPGdwmicieoBJ3wjtmwvfwDvGGzfu3kB+RD5mDphZq+gPHz4sXTgTwvep6fheNQ3f\np6Yx5PtkkmV/q/IWFioWYv5X8xE1NgofT/kYnew61dmOf+Gahu9T0/G9ahq+T03Dsn+IhHMJkG+X\nw9rSGspIJca7j5c6EhGR0TOZY/YlFSVYmrIUOZdz8MnUTxDQM0DqSEREJsMor7MnIqJHZ1LX2RvZ\nvz1ERK2CyR2zJyKiR8eyJyIyAyx7IiIzYDZlb2wPOTc2K1asgIeHB3x8fDB16lSUl5dLHcnopKSk\noH///ujbty/+8Y9/SB3HKF26dAnPPvssBgwYALlcjqioKKkjGTWNRgNfX19MnDixxfdlFmVvbA85\nN0bPPfccTp8+jby8PLi7u+Odd96ROpJR0Wg0ePnll5GSkoIzZ84gPj4eZ8+elTqW0bGxscF7772H\n06dPIzs7G++//z7fp4fYsmULPD09DXIVolmU/e8POaeGBQYGwtLyt78OQ4cORVFRkcSJjMuxY8fg\n5uaGnj17wsbGBrNmzcL+/fuljmV0unbtioEDBwIAZDIZPDw8cOXKFYlTGaeioiIkJSUhLCzMIFch\ntvqyb+pDzul/YmNjMW7cOKljGJXLly/D1dVV99rFxQWXL1+WMJHxu3jxInJzczF06FCpoxilZcuW\nYdOmTbohq6UZ3XX2zaGPh5ybg4bep/Xr1+uOGa5btw5t2rTBCy+8YOh4Ro2/7PdoVCoVpk+fji1b\ntkAmk0kdx+gcOHAAXbp0ga+vr8Huj9Mqyj41NbXe5UqlEhcuXICPjw+A3/7bNHjw4DoPOTcXDb1P\nv9u9ezeSkpKQlpZmoESmo1u3brh06ZLu9aVLl+Di4iJhIuNVU1ODadOm4cUXX8TkyZOljmOUjh49\nioSEBCQlJaGyshK3b9/G3LlzERcX12L7NLrbJbSkhh5yTr9dafJ///d/yMjIQKdOde8gau7UajX6\n9euHtLQ0ODs7Y8iQIYiPj4eHB59rfD8hBObNmwdHR0e89957UscxCRkZGfjnP/8JhULRovtp9cfs\n78f/ijdsyZIlUKlUCAwMhK+vLyIjI6WOZFSsra2xbds2BAUFwdPTE3/6059Y9PXIysrCJ598gvT0\ndPj6+sLX1xcpKSlSxzJ6hugms5rsiYjMlVlN9kRE5oplT0RkBlj2RERmgGVPRGQGWPZEjcjJyYGP\njw+qqqpQUVEBuVyOM2fOSB2L6JHwahyiJli9ejUqKytx7949uLq6YuXKlVJHInokLHuiJqipqYGf\nnx+eeOIJfPfdd/ydDTI5PIxD1AQ3btxARUUFVCoV7t27J3UcokfGyZ6oCSZNmoQXXngBv/zyC65e\nvYqtW7dKHYnokbSKG6ERtaS4uDi0bdsWs2bNglarxfDhw3H48GEEBARIHY2oyTjZExGZAR6zJyIy\nAyx7IiIzwLInIjIDLHsiIjPAsiciMgMseyIiM/D/AdOk4sTib7HqAAAAAElFTkSuQmCC\n"
      }
     ],
     "prompt_number": 63
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "\n"
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## R-Squared or Coefficient of Determination\n",
      "https://www.khanacademy.org/math/probability/regression/regression-correlation/v/r-squared-or-coefficient-of-determination\n",
      "\n",
      "If we have n points with x and y coordinates. $(x_1, y_1)\\cdots(x_n, y_n)$ .\n",
      "\n",
      "$$\n",
      "\\begin{align*}\n",
      "SE_{line} =  (y_1 - (mx_1 + b))^2 + (y_2 - (mx_2 + b))^2 +\\cdots + (y_n - (mx_n + b))^2\n",
      "\\end{align*}\n",
      "$$\n",
      "\n",
      "Now we can calculate the error.\n",
      "\n",
      "Now we want to come up with an estimate of how well the line fits the data. \n",
      "\n",
      "How much (what percentage) of the variation in $y$ is described by the variation in $x$ (or described by the line)?\n",
      "\n",
      "Start by find the total variation of $y$.\n",
      "\n",
      "\n",
      "total vairation in $y$ :\n",
      "$$\n",
      "\\begin{align*}\n",
      "(y_1 - \\bar{y})^2 + (y_2 - \\bar{y})^2 + \\cdots + (y_n - \\bar{y})^2 = SE_\\bar{y}\n",
      "\\end{align*}\n",
      "$$\n",
      "\n",
      "If you divide by $n$, you will get what we associate as the \"variance of $y$\".\n",
      "\n",
      "How much of the total variation is not described by the regression line? We have a measure of that, the squared error of the line, $SE_{line}$ .\n",
      "$$\n",
      "\\begin{align*}\n",
      "\\frac {SE_{line}} {SE_\\bar{y}}\n",
      "\\end{align*}\n",
      "$$\n",
      "\n",
      "What percentage of total variation is described by the line?\n",
      "$$\n",
      "\\begin{align*}\n",
      "r^2 = 1 - \\frac {SE_{line}} {SE_\\bar{y}}\n",
      "\\end{align*}\n",
      "$$\n",
      "aka: Coeffecient of Determination, or R-squared\n",
      "\n",
      "If $SE_{line}$ is small, it tells us that the line is a good fit. If the $SE_{line}$ is large, $r^2$ will be close to zero."
     ]
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "##Calculating R-Squared\n",
      "https://www.khanacademy.org/math/probability/regression/regression-correlation/v/calculating-r-squared\n",
      "\n",
      "For each of our points, we want to find the predicted $y$ based on each $x$ for our calculated line that minimizes the squared distance."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "m = (np.mean(x) * np.mean(y) - np.mean(x*y)) / (np.mean(x)**2 - np.mean(x**2))\n",
      "b = np.mean(y) - m * np.mean(x)\n",
      "se_line = 0\n",
      "se_mean_y = 0\n",
      "print m, b\n",
      "for i, a in enumerate(x):\n",
      "    y_star = m * a + b\n",
      "    se_w_line = (y[i] - y_star)**2\n",
      "    se_line += se_w_line\n",
      "    sqrd_distance_y_mean = (y[i] - np.mean(y))**2\n",
      "    se_mean_y += sqrd_distance_y_mean\n",
      "    print a, y[i], y_star, se_w_line, sqrd_distance_y_mean\n",
      "    \n",
      "print se_line\n",
      "print se_mean_y"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "stream",
       "stream": "stdout",
       "text": [
        "0.97619047619 -0.238095238095\n",
        "-2 -3 -2.19047619048 0.655328798186 10.5625\n",
        "-1 -1 -1.21428571429 0.0459183673469 1.5625\n",
        "1 2 0.738095238095 1.59240362812 3.0625\n",
        "4 3 3.66666666667 0.444444444444 7.5625\n",
        "2.7380952381\n",
        "22.75\n"
       ]
      }
     ],
     "prompt_number": 33
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "If we want to know the percentage of the total variation, not explained by the variation in x, we divide the total standard error with regard to the line by the total distance from the mean."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "se = se_line / se_mean_y\n",
      "display(Math(latex('\\\\frac {SE_{line}} {SE_\\\\bar{y}} = %0.2f'% se)))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "latex": [
        "$$\\frac {SE_{line}} {SE_\\bar{y}} = 0.12$$"
       ],
       "output_type": "display_data",
       "text": [
        "<IPython.core.display.Math object at 0x1040564d0>"
       ]
      }
     ],
     "prompt_number": 69
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "display(Math(latex('r^2 = 1 - \\\\frac {SE_{line}} {SE_\\\\bar{y}}')))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "latex": [
        "$$r^2 = 1 - \\frac {SE_{line}} {SE_\\bar{y}}$$"
       ],
       "output_type": "display_data",
       "text": [
        "<IPython.core.display.Math object at 0x105bedf50>"
       ]
      }
     ],
     "prompt_number": 72
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "r_sqr = 1 - se\n",
      "display(Math(latex('r^2 = %0.2f' % r_sqr)))"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "latex": [
        "$$r^2 = 0.88$$"
       ],
       "output_type": "display_data",
       "text": [
        "<IPython.core.display.Math object at 0x105c84610>"
       ]
      }
     ],
     "prompt_number": 71
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "88% of the variation in these $y$ values is explained by the line, or the variation in $x$. All of the points are closer to our actual line (green) than to the mean (blue)."
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [
      "plt.figure()\n",
      "ax1 = plt.axes(frameon=True)\n",
      "ax1.get_xaxis().tick_bottom()\n",
      "ax1.get_yaxis().tick_left()\n",
      "plt.axis([-4.5, 4.5, -4.5, 4.5])\n",
      "plt.plot(x, y, 'ro')\n",
      "t = np.arange(x[0] - x[-1])\n",
      "# y = b + m * x\n",
      "plt.plot([-10, 0, 10], [(b + m * -10), b, (b + m * 10)], 'g')\n",
      "#plt.plot(x, y, 'rx')\n",
      "plt.plot([-10, 10], [np.mean(y), np.mean(y)], 'b')\n",
      "plt.xlabel('x')\n",
      "plt.ylabel('y')\n",
      "plt.title('m = %f, b = %f' % (m, b))\n",
      "plt.show()"
     ],
     "language": "python",
     "metadata": {},
     "outputs": [
      {
       "output_type": "display_data",
       "png": "iVBORw0KGgoAAAANSUhEUgAAAXsAAAETCAYAAADXmaY8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XtYVHX+B/A3NyUcMUHFELyLggOIgpprSj8lzNt6X8m8\nASpQWvpb1y66uW2aq5s9oglasmYl1a6WjFyUCBExEg2RUdMiDVBRFEQH5TIz398fbfMTuYrDnBnm\n/Xoen2fmnDPz/XDG582H7zlzjoUQQoCIiFo1S6kLICKilsewJyIyAwx7IiIzwLAnIjIDDHsiIjPA\nsCciMgMMe6LHtGDBAqxZs0bqMogaxLCnx7Z371706NEDMpkMU6dORWlpab3bHj9+HEOHDoW9vT28\nvb2RkZGhW7d+/Xq0b99e98/Ozg5WVlYoKSnRbfPNN99g8ODBkMlkcHV1xb///W/dusWLF2PAgAGw\nsrLCxx9/XGPcyspKLF++HN26dYODgwNeeuklqNVq3fqSkhJMnToVMpkMPXv2RGxsbJN/fgsLC1hY\nWDR5e315lJo//vhj+Pr6okOHDnB1dcWqVaug0Wh061988UU89dRTsLe3R+/evbFu3boar//6668x\ncOBA2NvbY+DAgThw4ECN9atWrUKnTp3QqVMnvPbaazXWNfSZHzlyBJaWljU+908++eRxdgvVRxA9\nBqVSKdq3by/S09OFSqUSL7zwgpg9e3ad2966dUs4ODiI//znP0Kr1YpPP/1UdOzYUZSWlta5/dq1\na8WYMWN0z8+ePSu6dOkikpKShEajESUlJSIvL0+3/oMPPhApKSnC19dXfPzxx7Xea9SoUaK0tFQU\nFxeL4cOHi7feeku3fvbs2WL27NmivLxcHDt2THTo0EGcPXu2SftgwYIFYvXq1U3aVp8epeaoqChx\n7NgxUV1dLa5cuSKGDBkiNmzYoFuvVCrF/fv3hRBC/Pjjj8LJyUkkJiYKIYS4fv26sLOzE0lJSUII\nIeLj44WdnZ0oLi4WQggRHR0t+vfvL65cuSKuXLkiPDw8RHR0tBCi8c88NTVVuLi4tMwOohoY9q1I\njx49xKZNm4Snp6eQyWQiODhYFBUViXHjxgl7e3sxduzYeoO1uV5//XUxZ84c3fO8vDzRpk0boVKp\nam2rUCiEh4dHjWVubm5i165dtbbVarWiV69eYs+ePbplQUFB4q9//WujNY0cObJW2Pv6+op///vf\nuud79+4Vrq6uQgghVCqVaNOmjfjpp5906+fNmydee+21RscS4rewDwsLEwEBAaJ9+/Zi9OjR4tdf\nf23Sa5vrcWvevHmzmDRpUp3rfvzxR9GtWzdx6tQpIYQQGRkZokuXLjW26dy5s8jMzBRCCPH000+L\nDz/8ULcuJiZGDB8+XAjR+GfOsDccTuO0IhYWFti/fz9SUlJw4cIFHDx4EM8//zw2bNiAGzduQKvV\nIjIyss7X5ufno2PHjvX++/zzz+t83blz5+Dt7a173rt3b7Rt2xYXL15sUs1arRZnz56ttTw9PR3F\nxcWYPn26btn3338PIQS8vLzg7OyMuXPnNjhl9DDxwJVBtFotCgsLcffuXVy8eBHW1tbo27evbr23\nt3edddX3vp999hn++te/4ubNmxg0aBDmzJlT7/ZPPvlkvft548aNTRrzcWtOS0uDXC6vsSwiIgLt\n2rXDwIEDsXr1agwePFj3vtbW1jh48CA0Gg2+/vpr2NrawsvLC0Dt/wNeXl4N1vHwZ37jxg107doV\nvXv3xooVK3Dv3r0m/Qz0iCT+ZUN61LNnT7F3717d8+nTp4uIiAjd861bt4opU6bodcwxY8aIHTt2\n1FjWrVs3kZaWVmvbmzdvio4dO4rPP/9cVFVVid27dwtLS0sRFhZWa9vg4GCxcOHCGstsbGxEr169\nxE8//SRUKpWYPn16jb8qfldXZ7969Wrxhz/8QRQXF4tr166JoUOHCktLS1FUVCSOHj0qunbtWmP7\nnTt3Cn9//ybtg/nz54ugoCDdc5VKJaysrERhYWGTXt8cj1Pzrl27hKurq7h161atdVqtVqSmpgpH\nR0fx/fff65YrFAphZ2cnrK2thZ2dnYiPj9ets7KyEhcuXNA9v3jxorCwsBBCNP6ZFxUVifPnzwsh\nhLh06ZIYNWqUWLJkySPsCWoqdvatjJOTk+7xE088UeO5ra0tVCqVXseTyWQoKyursaysrAzt27ev\nta2joyO+/vprvPfee+jatSsOHTqEsWPHwsXFpcZ29+7dw3/+8x/Mnz+/xnI7OzssXLgQffv2Rbt2\n7fDGG28gISGhSXW++eab8PHxwaBBgzBy5EhMnToV1tbWcHJygkwmw507d5r0M9TFwsKixs/Qrl07\nODg44OrVq016fVM8//zzugOYsbGxza7566+/xhtvvIHExEQ4ODjUWm9hYQF/f3/MnDlTd8D3hx9+\nwOLFi5Geno7q6mqkpaUhNDQUZ86cAYBatZSVlUEmkwFo/DN3cnLCgAEDAAA9e/bExo0bsW/fvmbu\nJWoIw76VE028qGl+fn6NMyIe/lffmR4DBw5ETk6O7nleXh6qqqrg5uZW5/ajRo3CiRMncOvWLezZ\nswc//vgjhg4dWmObr776Co6Ojhg9enSN5b9PGzSHra0ttm7disLCQvz8889wcHCAr68vAMDNzQ1q\ntRo///yzbvucnJxa0xwNKSgo0D1WqVQoKSmBs7NzndvKZLJ69/OGDRvqfE1iYiLu3r2Lu3fvIigo\nqFk1JyUlYfHixTh48CAGDhzY4M9TXV2Ndu3aAQBSUlIwfPhw3bSOr68vhg0bhm+++QbAb/8HTp8+\nXW8dTfnMH6TVahusjZpJ6j8tSH969uwpUlJSdM9ffPFFsXbtWt3zDz/8UIwdO1avY549e1bY29vr\nzsYJCgqqMaXxsB9++EFUVVWJsrIy8corr4iRI0fW2iYgIKDGmTK/i4mJEb169RK//PKLKC8vFzNn\nzhTz5s3Tra+qqhL3798XI0aMEB9++KG4f/++0Gq1QgihO1NEq9WK7777Tri6uork5GTda2fPni2C\ngoJEeXm5SE9PFx06dBDnzp3TrbewsKhzakqI36Zx7O3txbFjx0RlZaV49dVX6/y59K2xmh+UkpIi\nHBwcRHp6eq11N27cELGxsUKlUgm1Wi2SkpKEvb29OHHihBBCiEOHDolOnTqJ06dPCyF++wwdHR11\n+y86Olq4u7uLK1euiMLCQuHh4VFjaq+hzzw1NVVcvnxZaLVakZ+fL0aPHi2Cg4P1to/o/zHsW5G6\nwv5vf/ub7vlHH30kAgIC9D7u3r17Rffu3UW7du3ElClTapzxExYWVmNOPigoSHTo0EF06NBBzJ49\nW3f63u8KCwuFjY1NjVMqH/TWW2+Jzp07i86dO4t58+aJ27dv69aNHj1aWFhYCEtLS2FhYVEjoI8e\nPSp69uwp7OzsxIABA2oc2xBCiJKSEjFlyhTRrl070aNHDxEbG6tbl5+fL+zt7UVJSUmdNS1YsECE\nh4eLgIAAIZPJxOjRo8Xly5ebuPear6Gaf/31VyGTyURBQYEQQohnn31W2NjYCJlMpvs3fvx4IYQQ\nxcXFYvTo0eLJJ58UHTp0EH5+fuLAgQM1xtq4caPo3bu3kMlkonfv3mLz5s011v/lL38RDg4OwsHB\nQaxatarGuoY+882bN4tu3boJOzs74erqKl555ZU6z+Six2chBG9eQtSQzz77DOfOnav1RSMiU8Kw\nJyIyAzxAS0RkBhj2RERmwFrqAh4mxQWliIhag4Zm5Y0u7IGmnxtORES/aaxR5jQOEZEZYNgTEZkB\nhj0RkRlg2BMRmQGGPRGRGWDYExGZAYY9EZEZkCTsNRoNfHx8MGnSJCmGJyJqNdRaNTZmNH47S0m+\nVLVlyxZ4eHjg7t27UgxPRNQq5BTlIDguGB1tOza6rcE7+8LCQiQkJCA0NJTflCUiaoZKdSXWpK5B\nwCcBeMnvJSTPTW70NQbv7JcvX45NmzbVun/mg9auXat77O/vD39//5YvjIjIBGQWZiL4QDDcHN1w\nOuw0nNvXffvLhxk07A8ePIguXbrAx8cHR44cqXe7B8OeiIiA8qpyrE5djc+Vn2PLuC2Y6THzkS4c\nadBpnOPHjyMuLg69evVCUFAQvv32W8ybN8+QJRARmZyUX1LgGeWJm/duIjc8F7MGznrkKwRLdqeq\ntLQ0/POf/4RCoahZkIUF5/KJiADcrriNlckrcejnQ4iaEIUJbhPq3bax7JT0PHteu56IqG5xF+Ig\n3y6HtaU1lBHKBoO+KYzuHrTs7InInBWXF2Np4lKcvHoSH03+CP49/Zv0OqPu7ImI6DdCCOzN3QvP\nKE+4dnDFmfAzTQ76pjDKO1UREZmTwjuFCDsYhl/LfoUiSAG/bn56H4OdPRGRRLRCi52ndsJnhw/8\nnP1wavGpFgl6gJ09EZEkfi75GYsUi3Cv+h5S56dC3kXeouOxsyciMiCNVoP3jr+H4R8Nx8R+E3E8\n+HiLBz3Azp6IyGCUN5QIiQuBnY0dMkMz0dehr8HGZmdPRNTCqjRV+NuRv+HZj59FiE8IUualGDTo\nAXb2REQtKutKFoLjgtGjQw9kL8mGi72LJHUw7ImIWsC96nt468hb+CTnE2wO3IwgeZCkVw1g2BMR\n6Vna5TSEKkLh6+yLM+Fn0KVdF6lLYtgTEenLnco7WPXNKiguKLB9wnZM7j9Z6pJ0eICWiEgPEn5K\ngHy7HGqtGsoIpVEFPcDOnojosdy8dxPLDy1HRn4G/vXHf2FM7zFSl1QndvZERM0ghMCXZ7+EZ5Qn\nOtl1Qm54rtEGPcDOnojokV29exUvJbyECzcvYP+s/Xja9WmpS2oUO3sioiYSQiAmOwaDogdB3kWO\n7CXZJhH0ADt7IqImuVR6CYsPLkbp/VIkz02Gd1fvZr/X0fh4HI6MhHVlJdRt2+K5ZcswasLj3Ymq\nMQx7IqIGaLQafJD1Ad5Oext/+cNfsOLpFbC2bH50Ho2Px6FXXsG6vDzdsjf/+7glA5+3JSQiqsf5\n4vMIiQuBlaUVdk3eBTdHt8d+z9WBgXjn8OFay9cEBuLvSUnNfl/elpCI6BFVa6qx7ug6PPOvZ/Ci\n14tIW5Cml6AHAOvKyjqXW1VU6OX96x23Rd+diMjE/HDtB4TEhcCpnRNOLT6FHk/20Ov7q9u2rXO5\nxtZWr+M8jJ09ERGACnUFXk95HeM+HYflw5cjcU6i3oMeAJ5btgxv9ulTY9kbffogYOlSvY/1IHb2\nRGT2juUfQ0hcCLycvHAm/Ay6yrq22Fi/H4Rds3UrrCoqoLG1xbilS1v8bBweoCUis3W38i7e+PYN\n7Du3D9vGb8M092lSl9RsPEBLRFSHw3mH4RnlCVWVCsoIpUkHfVNwGoeIzErp/VKsOLwCqZdSsWPi\nDgT2DZS6JINgZ09EZmP/+f2QR8khayNDbniu2QQ9wM6eiMxAkaoILye8DOUNJb6Y8QVGdh8pdUkG\nx86eiFotIQT25OyBd7Q3+jn2w+mw02YZ9AA7eyJqpfLL8rHk4BJcu3sNiXMSMfipwVKXJCl29kTU\nqmiFFtuztmPIziF4pvszyFqUZfZBD7CzJ6JW5OKtiwiNC4Vaq8bRBUfh3tld6pKMBjt7IjJ5aq0a\nGzM2YsSuEZjhMQPpC9MZ9A9hZ09EJi2nKAchcSHo+ERHZC3KQq+OvaQuySgx7InIJFWqK/FO+jvY\ncXIHNozdgIWDFsLCwkLqsowWw56ITE5mYSaCDwTDzdENp8NOw7m9s9QlGT2GPRGZjPKqcqxOXY3P\nlZ9jy7gtmOkxk918E/EALRGZhJRfUuAZ5Ymb924iNzwXswbOYtA/Anb2RGTUblfcxsrklTj08yFE\nTYjCBLeWve57a8XOnoiMVtyFOMi3y2FtaQ1lhJJB/xjY2ROR0SkuL8aypGU4efUkPpv2GUb3HC11\nSSbP4J19QUEBnn32WQwcOBByuRyRkZGGLoGoVTgaH4/VgYFY6++P1YGBOBofL3VJj00Igb25e+EZ\n5QkXexfkhOUw6PXE4J29jY0N3n//fQwaNAgqlQpDhgxBQEAA3N35bTeipjoaH49Dr7yCdXl5umVv\n/vdxS9/LtKUU3ilEeHw4Lt++DEWQAn7d/KQuqVUxeGfftWtXDBo0CAAgk8ng7u6Oq1evGroMIpN2\nODKyRtADwLq8PCRv3SpRRc2nFVrsPLUTPjt84Ofsh1OLTzHoW4Ckc/aXL19GdnY2hg0bVmP52rVr\ndY/9/f3h7+9v2MKIjJx1ZWWdy60qKgxcyePJK8lDqCIU96rvIXV+KuRd5FKX1GpJFvYqlQozZszA\nli1bIJPJaqx7MOyJqDZ127Z1LtfY2hq4kubRaDXY8v0WrE9fj9dHvo5Xh78KK0srqctq1SQJ++rq\nakyfPh0vvvgipkyZIkUJRCbtuWXL8GZeXo2pnDf69MG4pUslrKpplDeUCIkLgZ2NHTJDM9HXoa/U\nJZkFCyGEMOSAQgjMnz8fjo6OeP/992sXZGEBA5dEZJKOxscjeetWWFVUQGNri4ClS4364GyVpgrv\npr+LbVnbsO5/1iF0cCgsLfhVH31pLDsNHvbHjh3DqFGj4OXlpfuq87vvvotx48b9VhDDnqjVybqS\nhZC4EHTv0B3RE6PhYu8idUmtjtGFfWMY9kStx73qe1h7ZC325OzB5sDNCJIH8Xo2LaSx7OQ3aImo\nRaRdTkOoIhS+zr44E34GXdp1kboks8awJyK9ulN5B6u+WQXFBQW2T9iOyf0nS10SgRdCIyI9Svgp\nAZ5RnlBr1VBGKBn0RoSdPRE9tlv3buHVQ68iIz8DMZNjMKb3GKlLooewsyeiZhNC4MuzX0IeJUcn\nu07IDc9l0BspdvZE1CxX717FSwkv4cLNC/jqT19huMtwqUuiBrCzJ6JHIoRATHYMBkUPgryLHNlL\nshn0JoCdPRE12aXSS1h8cDFK75cieW4yvLt6S10SNRE7eyJqlEarQeT3kfD70A8BvQOQGZrJoDcx\n7OyJqEHni88jVBEKKwsrHA85DjdHN6lLomZgZ09EdarWVGN9+nqM2j0Kczzn4MiCIwx6E8bOnohq\nyb6WjeC4YDi1c8LJRSfR48keUpdEj4lhT0Q6FeoKvJ32NnZl78KmgE2Y6zWXFy5rJRj2RAQAyMjP\nQEhcCDydPHEm7AycZE5Sl0R6xLAnMnOqKhVeT3kd+87tw7bx2zDNfZrUJVEL4AFaIjN2OO8wPKM8\noapSQRmhZNC3YuzsicxQ6f1SrDi8AqmXUrFj4g4E9g2UuiRqYezsiczM/vP7IY+SQ9ZGhtzwXAa9\nmWBnT2Qmrquu4+XEl5F7PRdfzPgCI7uPlLokMiB29kStnBACe3L2wCvaC30d+uJ02GkGvRliZ0/U\niuWX5WPJwSUoUhUhcU4iBj81WOqSSCLs7IlaIa3QYnvWdgzZOQTPdH8GJ0JPMOjNHDt7olbm4q2L\nCI0LhVqrxtEFR+He2V3qksgIsLMnaiXUWjU2ZmzEiF0jMMNjBtIXpjPoSYedPVErkFOUg5C4EHR8\noiOyFmWhV8deUpdERoZhT2TCKtWVeCf9Hew4uQMbxm7AwkELeeEyqhPDnshEZRZmIiQuBG6Objgd\ndhrO7Z2lLomMGMOeyMSUV5VjTeoaxCpjsWXcFsz0mMlunhrFA7REJuTbS9/CK9oLxfeKoQxXYtbA\nWQx6ahJ29kQmoKyiDH9O/jMO/XwI0ROjMb7feKlLIhPDzp7IyCkuKCCPksPa0hrKCCWDnpqFnT2R\nkSouL8aypGU4efUkPp36KUb3HC11SWTC2NkTGRkhBGJzY+EZ5QkXexfkhOUw6OmxsbMnMiKFdwoR\nHh+OX2//CkWQAn7d/KQuiVoJdvZERkAIgZ2ndsJnhw/8nP1wcvFJBj3pFTt7IonlleRhkWIRyqvL\nkTo/FfIucqlLolaInT2RRDRaDTZ/txnDPhqGiW4TcTz4OIOeWoyFEEJIXcSDfvuCiFGVRERkAizQ\nUJwb5TSOcf36IdKfKk0VNhzbgK0ntmLd/6xD6OBQWFrwD2x6fI19kdoow56oNcq6koWQuBB079Ad\n2Uuy4WLvInVJZEYkaSmSkpIwYMAA9OvXD//4xz+kKIHIYO5X38dfkv+CSbGT8NrI16AIUjDoyeAM\nPmev0WjQv39/fPPNN+jWrRv8/PwQGxsLd/ff7qhjYdHwvBORKUm7nIZQRSh8nX0ROS4Sndt1lrok\naqUay856O/vIyEiUlpbqvaATJ06gb9++6NmzJ2xsbDB79mwcOHBA7+MQSelO5R2Ex4djzv45eO+5\n9xA7PZZBT5Kqd87++vXr8PPzw+DBgxEcHIzAwEC9XEr1ypUrcHV11T13cXHB999/X2ObtWvX6h77\n+/vD39//scclMpTEnxIRFh+G5/o8B2WEEk/aPil1SUT1h/26devw97//HYcPH8bu3bvx8ssvY9as\nWQgJCUGfPn2aPWBTfmE8GPZEpuLWvVtYfmg5MgoyEDM5BmN6j5G6JCKdBg/QWlpaomvXrnBycoKV\nlRVKS0sxY8YMrFy5stkDduvWDQUFBbrnBQUFcHHhwSoyXUIIfHn2S8ij5HC0c8SZsDMMejI69R6g\n3bJlC/bs2QNHR0eEhoZi6tSpsLGxgVarRb9+/ZCXl9esAdVqNfr374+UlBQ4Oztj6NChPEBLJuva\n3WuISIjAhZsXEPPHGAx3GS51SWSmGsvOeqdxSkpKsH//fvTo0aPGcktLSygUimYXZG1tjW3btiEw\nMBAajQYhISG6oCcyFUII7D69G6u+WYUw3zB8Pv1ztLVuK3VZRPUyysslGFlJRDVcvn0ZixWLUXK/\nBLsm74J3V2+pSyJq/qmXRFSTVmgR+X0k/D70w9jeY5EZmsmgJ5PByyUQNcH54vMIVfx2HZuM4Ay4\nObpJXRLRI2FnT9SAak011qevx6jdozDHcw7SFqQx6MkksbMnqkf2tWwExwWjq6wrTi46iR5P9mj8\nRURGimFP9JAKdQXeTnsbu7J3YVPAJsz1mquXb48TSYlhT/SAjPwMhMSFwNPJE2fCzsBJ5iR1SUR6\nwbAnAqCqUuGNlDew7/w+bH1+K6a5T5O6JCK94gFaMnuH8w7DM8oTd6vuQhmuZNBTq8TOnsxW6f1S\nrDi8AqmXUrFj4g4E9g2UuiSiFsPOnszSV+e/gjxKDlkbGXLDcxn01Oqxsyezcl11HS8nvozc67n4\nYsYXGNl9pNQlERkEO3syC0II7MnZA69oL/Rz6IfTYacZ9GRW2NlTq5dflo8lB5egSFWExDmJGPzU\nYKlLIjI4dvbUammFFtuztmPIziF4pvszOBF6gkFPZoudPbVKF29dRGhcKNRaNY4uOAr3zrxnApk3\ndvbUqqi1amzM2IgRu0ZghscMpC9MZ9ATgZ09GZmj8fE4HBkJ68pKqNu2xXPLlmHUhAlNeu2Z62cQ\nfCAYHZ/oiKxFWejVsVcLV0tkOhj2ZDSOxsfj0CuvYN0D9zd+87+PGwr8SnUl1qWvQ/TJaGwYuwEL\nBy3khcuIHsJpHDIahyMjawQ9AKzLy0Py1q31viazMBODdw5G7o1cnA47jWCfYAY9UR3Y2ZPRsK6s\nrHO5VUVFrWXlVeVYk7oGscpYbBm3BTM9ZjLkiRrAzp6Mhrpt2zqXa2xtazz/9tK38Ir2QvG9YijD\nlZg1cBaDnqgR7OzJaDy3bBnezMurMZXzRp8+GLd0KQCgrKIMf07+Mw79fAjRE6Mxvt94qUolMjkM\nezIavx+EXbN1K6wqKqCxtcW4pUsxasIEKC4oEJEQgUluk6CMUMK+rb3E1RKZFgshhJC6iAdZWFjA\nyEoiCRWXF2NZ0jKcvHoSH036CKN7jpa6JCKj1Fh2cs6ejJIQArG5sfCM8oSLvQtywnIY9ESPgdM4\nZHQK7xQiPD4cv97+FYogBfy6+UldEpHJY2dPRkMIgZ2ndsJnhw/8nP1wcvFJBj2RnrCzJ6OQV5KH\nRYpFKK8uR+r8VMi7yKUuiahVYWdPktJoNdj83WYM+2gYJrpNxPHg4wx6ohbAzp4kc/bGWQTHBcPO\nxg6ZoZno69BX6pKIWi129mRwVZoqvJ32Nvw/9keITwhS5qUw6IlaGDt7MqisK1kIiQtB9w7dkb0k\nGy72LlKXRGQWGPZkEPer7+OtI29hT84ebA7cjCB5EK9nQ2RADHtqcWmX0xCqCIWvsy9yw3PRuV1n\nqUsiMjsMe2oxdyrvYNU3q6C4oMD2Cdsxuf9kqUsiMls8QEstIvGnRHhGeUKtVUMZoWTQE0mMnT3p\n1a17t7D80HJkFGQgZnIMxvQeI3VJRAR29qQnQgh8efZLyKPkcLRzxJmwMwx6IiPCzp4e27W71xCR\nEIELNy/gqz99heEuw6UuiYgews6emk0IgZjsGHhHe8Oziyeyl2Qz6ImMFDt7apbLty9jsWIxSu6X\nIHluMry7ektdEhE1wKCd/cqVK+Hu7g5vb29MmzYNZWVlhhye9ECj1SDy+0j47vTF2N5jkRmayaAn\nMgEGvS1hcnIyxowZA0tLS7z22msAgA0bNtQsiLclNFrni88jVBEKKwsrfDT5I7g5ukldEhH9l1Hd\nljAgIACWlr8NOWzYMBQWFhpyeGqmak011qevx6jdozDHcw6OLDjCoCcyMZLN2cfExCAoKKjOdWvX\nrtU99vf3h7+/v2GKolqyr2UjOC4YTu2ccHLRSfR4sofUJRFRM+h9GicgIABFRUW1lq9fvx6TJk0C\nAKxbtw4//PAD9u3bV7sgTuMYhQp1Bd5Oexu7sndhU8AmzPWaywuXERmxxrJT7519cnJyg+t3796N\nhIQEpKSk6Hto0pOM/AyExIXA08kTZ8LOwEnmJHVJRPSYDDqNk5SUhE2bNiEtLQ22traGHJqaQFWl\nwuspr2PfuX3YNn4bprlPk7okItITg56N069fP1RVVcHBwQEA8PTTT2P79u01C+I0jiQO5x3GkoNL\n4N/TH5uf24yOT3SUuiQiegSNZadBw74pGPaGVXq/FCsOr0DqpVTsmLgDgX0DpS6JiJrBqE69JOOy\n//x+yKNa6ErjAAAKyElEQVTkkLWRITc8l0FP1Irxcglm6LrqOl5OfBm513PxxYwvMLL7SKlLIqIW\nxs7ejAghsCdnD7yivdDXoS9Oh51m0BOZCXb2ZiK/LB9LDi5BkaoIiXMSMfipwVKXREQGxM6+ldMK\nLbZnbceQnUPwTPdncCL0BIOeyAyxs2/FLt66iNC4UKi1ahxdcBTund2lLomIJMLOvhVSa9XYmLER\nI3aNwAyPGUhfmM6gJzJz7OxbmZyiHITEhaDjEx2RtSgLvTr2krokIjICDPtWolJdiXfS38GOkzuw\nYewGLBy0kBcuIyIdhn0rkFmYiZC4ELg5uuF02Gk4t3eWuiQiMjIMexNWXlWONalrEKuMxZZxWzDT\nYya7eSKqEw/QmqhvL30Lr2gvFN8rhjJciVkDZzHoiahe7OxNTFlFGf6c/Gcc+vkQoidGY3y/8VKX\nREQmgJ29CVFcUEAeJYe1pTWUEUoGPRE1GTt7E1BcXoxlSctw8upJfDr1U4zuOVrqkojIxLCzN2JC\nCOzN3QvPKE+42LsgJyyHQU9EzcLO3kgV3ilEeHw4Lt++DEWQAn7d/PQ+xtH4eByOjIR1ZSXUbdvi\nuWXLMGrCBL2PQ0TSY9gbGSEEPvzhQ7z57ZtYOnQp9s3ahzZWbfQ+ztH4eBx65RWsy8vTLXvzv48Z\n+EStD29LaETySvKwSLEI5dXl2DV5F+Rd5C021urAQLxz+HCt5WsCA/H3pKQWG5eIWgZvS2gCNFoN\nNn+3GcM+GoaJbhNxPPh4iwY9AFhXVta53KqiokXHJSJpcBpHYmdvnEVwXDDsbOyQGZqJvg59DTKu\num3bOpdrbG0NMj4RGRY7e4lUaarwdtrb8P/YHyE+IUiZl2KwoAeA55Ytw5t9+tRY9kafPghYutRg\nNRCR4bCzl0DWlSyExIWge4fuyF6SDRd7F4PX8PtB2DVbt8KqogIaW1uMW7qUB2eJWikeoDWge9X3\nsPbIWuzJ2YPNgZsRJA/i9WyISC8ay0529gaSdjkNoYpQ+Dr74kz4GXRp10XqkojIjDDsW9idyjtY\n9c0qKC4osH3CdkzuP1nqkojIDPEAbQtK+CkBnlGeUGvVUEYoGfREJBl29i3g1r1bePXQq8jIz0DM\n5BiM6T1G6pKIyMyxs9cjIQS+PPsl5FFydLLrhNzwXAY9ERkFdvZ6cu3uNUQkRODCzQv46k9fYbjL\ncKlLIiLSYWf/mIQQiMmOgXe0N+Rd5Mheks2gJyKjw87+MVwqvYTFBxej9H4pkucmw7urt9QlERHV\niZ19M2i0GkR+Hwm/D/0Q0DsAmaGZDHoiMmrs7B/R+eLzCFWEwsrCCsdDjsPN0U3qkoiIGsXOvomq\nNdVYn74eo3aPwhzPOTiy4AiDnohMBjv7Jsi+lo3guGA4tXPCyUUn0ePJHlKXRET0SBj2DahQV+Dt\ntLexK3sXNgVswlyvubxwGRGZJIZ9PTLyMxASFwJPJ0+cCTsDJ5mT1CURETUbw/4hqioVXk95HfvO\n7cO28dswzX2a1CURET02HqB9wOG8w5Bvl0NVpYIyQsmgJ6JWQ5Kwf++992BpaYmSkhIphq+l9H4p\nFh5YiMWKxdgxcQf+9cd/weEJB6nLIiLSG4OHfUFBAZKTk9Gjh3Gc0bL//H7Io+SQtZEhNzwXgX0D\npS6JiEjvDD5nv2LFCmzcuBF//OMfDT10DUWqIixNXIrc67n4YsYXGNl9pKT1EBG1JIOG/YEDB+Di\n4gIvL68Gt1u7dq3usb+/P/z9/fVWgxACn5z5BCuTVyLYJxifTP0Etta2ent/IiJjpPewDwgIQFFR\nUa3l69atw7vvvovDhw/rltV3c9wHw16f8svyseTgEhSpipA4JxGDnxrcIuMQERkbC9HQ7cj1SKlU\nYsyYMbCzswMAFBYWolu3bjhx4gS6dPn/m283dof05tAKLaJPRuOtI29h+fDlWDliJWysbPQ6BhGR\nlBrLToOF/cN69eqFU6dOwcGh5lkv+g77i7cuIjQuFGqtGrsm74J7Z3e9vTcRkbFoLDslO8++pS87\noNaqsTFjI0bsGoEZHjOQvjCdQU9EZkuyzr4++ujsc4pyEBIXgo5PdMTOiTvRq2MvPVVHRGScGsvO\nVnW5hEp1Jd5Jfwc7Tu7AhrEbsHDQQl64jIgIrSjsMwszERIXgn4O/XA67DSc2ztLXRIRkdEw+bAv\nryrH6tTV+Fz5ObaM24KZHjPZzRMRPcSkL4T27aVv4RXthZv3biI3PBezBs6qEfRHjhyRrjgTwv3U\ndNxXTcP91DSG3E8mGfa3K25jkWIRFny9AJHjIvHJ1E/Qya5Tre34H65puJ+ajvuqabifmoZh34C4\nC3GQb5fD2tIaygglJrhNkLokIiKjZzJz9sXlxViWtAxZV7Lw6bRP4d/TX+qSiIhMhlGeZ09ERI/O\npM6zN7LfPURErYLJzdkTEdGjY9gTEZkBhj0RkRkwm7A3tpucG5uVK1fC3d0d3t7emDZtGsrKyqQu\nyegkJSVhwIAB6NevH/7xj39IXY5RKigowLPPPouBAwdCLpcjMjJS6pKMmkajgY+PDyZNmtTiY5lF\n2BvbTc6N0XPPPYezZ88iJycHbm5uePfdd6UuyahoNBq8/PLLSEpKwrlz5xAbG4vz589LXZbRsbGx\nwfvvv4+zZ88iMzMTH3zwAfdTA7Zs2QIPDw+DnIVoFmH/+03OqX4BAQGwtPztv8OwYcNQWFgocUXG\n5cSJE+jbty969uwJGxsbzJ49GwcOHJC6LKPTtWtXDBo0CAAgk8ng7u6Oq1evSlyVcSosLERCQgJC\nQ0MNchZiqw/7pt7knP5fTEwMxo8fL3UZRuXKlStwdXXVPXdxccGVK1ckrMj4Xb58GdnZ2Rg2bJjU\npRil5cuXY9OmTbomq6UZ3Xn2zaGPm5ybg/r20/r163VzhuvWrUObNm3wwgsvGLo8o8Yv+z0alUqF\nGTNmYMuWLZDJZFKXY3QOHjyILl26wMfHx2DXx2kVYZ+cnFzncqVSiUuXLsHb2xvAb382DRkypNZN\nzs1Fffvpd7t370ZCQgJSUlIMVJHp6NatGwoKCnTPCwoK4OLiImFFxqu6uhrTp0/Hiy++iClTpkhd\njlE6fvw44uLikJCQgIqKCty5cwfz5s3Dnj17WmxMo7tcQkuq7ybn9NuZJv/7v/+LtLQ0dOpU+wqi\n5k6tVqN///5ISUmBs7Mzhg4ditjYWLi7877GDxJCYP78+XB0dMT7778vdTkmIS0tDf/85z+hUCha\ndJxWP2f/IP4pXr+lS5dCpVIhICAAPj4+iIiIkLoko2JtbY1t27YhMDAQHh4e+NOf/sSgr0NGRgY+\n/fRTpKamwsfHBz4+PkhKSpK6LKNniGwyq86eiMhcmVVnT0Rkrhj2RERmgGFPRGQGGPZERGaAYU/U\niKysLHh7e6OyshLl5eWQy+U4d+6c1GURPRKejUPUBGvWrEFFRQXu378PV1dXrFq1SuqSiB4Jw56o\nCaqrq+Hr64snnngC3333Hb+zQSaH0zhETXDz5k2Ul5dDpVLh/v37UpdD9MjY2RM1weTJk/HCCy/g\nl19+wbVr17B161apSyJ6JK3iQmhELWnPnj1o27YtZs+eDa1WixEjRuDIkSPw9/eXujSiJmNnT0Rk\nBjhnT0RkBhj2RERmgGFPRGQGGPZERGaAYU9EZAYY9kREZuD/AL3X5k4nDJEMAAAAAElFTkSuQmCC\n"
      }
     ],
     "prompt_number": 47
    },
    {
     "cell_type": "markdown",
     "metadata": {},
     "source": [
      "## Covariance and the Regression Line\n",
      "https://www.khanacademy.org/math/probability/regression/regression-correlation/v/covariance-and-the-regression-line\n",
      "\n",
      "The covariance between two random variables, the product of the distance of each variable from their mean.\n",
      "\n",
      "$E$ signifies the expected value.\n",
      "\n",
      "$$Cov(x, y) = E[(x - E[x])(y - e[y])]$$\n",
      "\n",
      "Lets say we have one sample of $(x, y)$ . \n",
      "\n",
      "$x = 1$ , $E[x] = 0$ , $y = 3$ , $E[y] = 4$\n",
      "\n",
      "$$Cov(x, y) = -1$$\n",
      "\n",
      "This tells us that for this one random sample, $x$ was above its expected value while $y$ was below its expected value.\n",
      "\n",
      "We want to connect the covariance formula with the other math above.\n",
      "\n",
      "$E[E[x]] = E[x]$\n",
      "That is, expected value of the expected value simply equals the expected value.\n",
      "\n",
      "$$\n",
      "\\begin{align*}\n",
      "Cov(x, y) &= E[(x - E[x])(y - e[y])]\\\\\\\\\n",
      "&= E[xy - xE[y] - E[x]y + E[x]E[y]]\\\\\\\\\n",
      "&= E[xy] - E[xE[y]] - E[E[x]y] + E[E[x]E[y]]\\\\\\\\\n",
      "&= E[xy] - E[y]E[x] - E[x]E[y] + E[x]E[y]\\\\\\\\\n",
      "&= E[xy] - E[y]E[x]\n",
      "\\end{align*}\n",
      "$$\n",
      "\n",
      "$$\\begin{align*}\n",
      "E[xy] &\\approx \\bar{xy}\\\\\\\\\n",
      "E[x] &\\approx \\bar{x}\\\\\\\\\n",
      "E[y] &\\approx \\bar{y}\n",
      "\\end{align*}\n",
      "$$\n",
      "\n",
      "$$\n",
      "\\begin{align*}\n",
      "Cov(x, y) &= E[(x - E[x])(y - e[y])]\\\\\\\\\n",
      "&\\vdots \\\\\\\\\n",
      "&= E[xy] - E[y]E[x]\\\\\\\\\n",
      "&\\approx \\bar{xy} - \\bar{y}\\bar{x}\n",
      "\\end{align*}\n",
      "$$\n",
      "\n",
      "$$\n",
      "\\begin{align*}\n",
      "Cov(x, x) &\\approx \\bar{x^2} - (\\bar{x})^2\\\\\\\\\n",
      "&= \\bar{xx} - \\bar{x}\\bar{x}\\\\\\\\\n",
      "&= Var(x)\n",
      "\\end{align*}\n",
      "$$\n",
      "\n",
      "$\\hat{m}$ is the approximate slope. \n",
      "$$\n",
      "\\begin{align*}\n",
      "\\hat{m} &= \\frac {\\bar{xy} - \\bar{y}\\bar{x}} {\\bar{x^2} - (\\bar{x})^2}\\\\\\\\\n",
      "&= \\frac {Cov(x, y)} {Var(x)}\n",
      "\\end{align*}\n",
      "$$"
     ]
    },
    {
     "cell_type": "code",
     "collapsed": false,
     "input": [],
     "language": "python",
     "metadata": {},
     "outputs": []
    }
   ],
   "metadata": {}
  }
 ]
}