train.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "lines_to_next_cell": 0
   },
   "source": [
    "# Philly Evictions Model Training\n",
    "## Environment Setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import pipeline\n",
    "\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "\n",
    "pd.set_option('display.max_columns', None)\n",
    "pd.set_option('display.max_rows', None)\n",
    "\n",
    "pipeline.notebook.set_up()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load Data\n",
    "Our evictions data has already been augmented with data from the ACS and from\n",
    "Philadelphia's open data portal. We load in the final merged dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv('data/final_merged_df.csv')\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Split Data By Year\n",
    "We have data for 2009 to 2016. We want to split this data into training set /\n",
    "test set pairs using a temporal cross-validation approach."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "split_output_filename = 'results/time_splits/time_split_ay.csv'\n",
    "clf_output_filename = 'results/evaluation_results/clf-small-grid_ay.csv'\n",
    "reg_output_filename = 'results/evaluation_results/reg-small-grid_ay.csv'\n",
    "splits = pipeline.split_all_years(df, colname='year_evictions')\n",
    "\n",
    "split_table = pipeline.split_boundaries(splits, colname='year_evictions')\n",
    "split_table.to_csv(split_output_filename, index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Cleaning\n",
    "We want to clean each of our training set / test set pairs. We use a function\n",
    "called `clean_split()` that cleans both sets at once, making sure to clean the\n",
    "test data using the same bins and categories applied to the training data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "lines_to_next_cell": 0
   },
   "outputs": [],
   "source": [
    "%psource pipeline.clean_split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cleaned_splits = [pipeline.clean_split(split) for split in splits]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Labeling\n",
    "We plan to use both regression-based models and binary classifiers. For our\n",
    "binary classifiers, we will need to label our data using a binary label.\n",
    "\n",
    "Our binary label separates block groups into two classes: \"high\" and \"low\"\n",
    "eviction rate block groups. The \"high\" eviction rate block groups are those\n",
    "that we believe should be prioritized for intervention.\n",
    "\n",
    "Any block group with more than 14 evictions is considered a \"high\" eviction\n",
    "rate block group. Roughly 16% of Philadelphia block groups are \"high\" eviction\n",
    "rate block groups. We have picked this lower boundary because we know that\n",
    "Philadelphia can afford to target about 16% of block groups for intervention."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "labeled_splits = [pipeline.label(split, lower_bound=15, drop_column=True)\n",
    "                  for split in cleaned_splits]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lines_to_next_cell": 0
   },
   "source": [
    "## Model Generation\n",
    "### Binary Classifiers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "clfs = pipeline.clfs_small\n",
    "clfs_grid = pipeline.clf_small_grid"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lines_to_next_cell": 0
   },
   "source": [
    "Our binary classifiers are given by the following list:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "clfs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lines_to_next_cell": 0
   },
   "source": [
    "We plan to run a grid search using the following hyperparameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "clfs_grid"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lines_to_next_cell": 0
   },
   "source": [
    "We also want to evaluate our models at the following thresholds:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "thresholds = [14, 21, 28]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lines_to_next_cell": 0
   },
   "source": [
    "We run our models for each of our splits."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "results_df = pd.DataFrame(columns=[\n",
    "    'split',\n",
    "    'classifier',\n",
    "    'parameters',\n",
    "    'threshold'\n",
    "] + pipeline.evaluate.ClassifierEvaluator.metric_names())\n",
    "\n",
    "for i, (train_df, test_df) in enumerate(labeled_splits, start=1):\n",
    "    train_df = train_df.drop(columns=['GEOID', 'year_evictions'])\n",
    "    test_df = test_df.drop(columns=['GEOID', 'year_evictions'])\n",
    "    df = pipeline.run_clf_loop(\n",
    "        test_df, train_df, clfs, clfs_grid, 'label', thresholds, debug=False\n",
    "    )\n",
    "\n",
    "    df = df.assign(split=i)\n",
    "    results_df = results_df.append(df, ignore_index=True)\n",
    "\n",
    "results_df.to_csv(clf_output_filename, index=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lines_to_next_cell": 0
   },
   "source": [
    "### Regression Models"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "regs = pipeline.regs\n",
    "regs_grid = pipeline.reg_small_grid"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our regression models are given by the following list:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "regs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lines_to_next_cell": 0
   },
   "source": [
    "We plan to run a grid search using the following hyperparameters:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "regs_grid"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lines_to_next_cell": 0
   },
   "source": [
    "We run our models for each of our splits."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "results_df = pd.DataFrame(columns=[\n",
    "    'split',\n",
    "    'classifier',\n",
    "    'parameters',\n",
    "] + pipeline.evaluate.RegressionEvaluator.metric_names())\n",
    "\n",
    "for i, (train_df, test_df) in enumerate(cleaned_splits, start=1):\n",
    "    train_df = train_df.drop(columns=['GEOID', 'year_evictions'])\n",
    "    test_df = test_df.drop(columns=['GEOID', 'year_evictions'])\n",
    "    df = pipeline.run_reg_loop(\n",
    "        test_df, train_df, regs, regs_grid, 'evictions'\n",
    "    )\n",
    "\n",
    "    df = df.assign(split=i)\n",
    "    results_df = results_df.append(df, ignore_index=True)\n",
    "\n",
    "results_df.to_csv(reg_output_filename, index=False)"
   ]
  }
 ],
 "metadata": {
  "jupytext": {
   "cell_metadata_filter": "-all",
   "main_language": "python",
   "notebook_metadata_filter": "-all"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}