examples/grocery_sales/python/01_prepare_data/ojdata_preparation.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<i>Copyright (c) Microsoft Corporation.</i>\n",
    "\n",
    "<i>Licensed under the MIT License.</i>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data Preparation for Retail Sales Forecasting\n",
    "\n",
    "This notebook introduces how to split the Orange Juice dataset into training sets and test sets for training and evaluating different retail sales forecasting methods.\n",
    "\n",
    "We use backtesting a method that tests a predictive model on historical data to evaluate the forecasting methods. Other than standard [K-fold cross validation](https://en.wikipedia.org/wiki/Cross-validation_%28statistics%29) which randomly splits data into K folds, we split the data so that any of the time stamps in the training set is no later than any of the time stamps in the test set to ensure that no future information is used (expect certain information that we can know beforehand, e.g., price of the product in the next few weeks as we can set the price manually).\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Global Settings and Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%load_ext autoreload\n",
    "%autoreload 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "System version: 3.6.10 |Anaconda, Inc.| (default, Jan  7 2020, 21:14:29) \n",
      "[GCC 7.3.0]\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "import sys\n",
    "\n",
    "from fclib.common.utils import git_repo_path\n",
    "from fclib.dataset.ojdata import download_ojdata, split_train_test\n",
    "\n",
    "print(\"System version: {}\".format(sys.version))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Use False if you've already downloaded and split the data\n",
    "DOWNLOAD_DATA = True\n",
    "\n",
    "# Data directory\n",
    "DATA_DIR = os.path.join(git_repo_path(), \"ojdata\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Download Data\n",
    "\n",
    "We need to download the Orange Juice data before splitting it into training and test sets. By default, the following cell will download the data. If you've already done so, you may skip this part by switching `DOWNLOAD_DATA` to `False`.\n",
    "\n",
    "The dataset is from R package [bayesm](https://cran.r-project.org/web/packages/bayesm/index.html) and is part of the [Dominick's dataset](https://www.chicagobooth.edu/research/kilts/datasets/dominicks). It contains the following two csv files:\n",
    "\n",
    "1. `yx.csv` includes weekly sales of refrigerated orange juice at 83 stores. This files has 106139 rows and 19 columns. It contains weekly sales and prices of 11 orange juice brands as well as information about profit, deal, and advertisement for each brand. Note that the weekly sales is captured by a column named `logmove` which corresponds to the natural logarithm of the number of units sold. To get the number of units sold, you need to apply an exponential transform to this column.\n",
    "\n",
    "2. `storedemo.csv` includes demographic information on those stores. This table has 83 rows and 13 columns. For every store, the table describes demographic information of its consumers, distance to the nearest warehouse store, average distance to the nearest 5 supermarkets, ratio of its sales to the nearest warehouse store, and ratio of its sales\n",
    "to the average of the nearest 5 stores.\n",
    "\n",
    "Note that the week number starts from 40 in this dataset, while the full Dominick's dataset has week number from 1 to 400. According to [Dominick's Data Manual](https://www.chicagobooth.edu/-/media/enterprise/centers/kilts/datasets/dominicks-dataset/dominicks-manual-and-codebook_kiltscenter.aspx), week 1 starts on 09/14/1989.\n",
    "Please see pages 40 and 41 of the [bayesm reference manual](https://cran.r-project.org/web/packages/bayesm/bayesm.pdf) and the [Dominick's Data Manual](https://www.chicagobooth.edu/-/media/enterprise/centers/kilts/datasets/dominicks-dataset/dominicks-manual-and-codebook_kiltscenter.aspx) for more details about the data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Data already exists at the specified location.\n",
      "Data download completed. Data saved to /data/home/vapaunic/forecasting/ojdata\n"
     ]
    }
   ],
   "source": [
    "if DOWNLOAD_DATA:\n",
    "    download_ojdata(DATA_DIR)\n",
    "    print(\"Data download completed. Data saved to \" + DATA_DIR)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Parameters"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we define parameters for data preparation and forecast settings. In particular:\n",
    "- *N_SPLITS* defines the number of training/testing splits we want to split our data into\n",
    "- *HORIZON* or forecasting horizon determines the number of weeks to forecast in the future, or the test period length\n",
    "- *GAP* defines the gap (in weeks) between the training and test data. This is to allow business managers to plan for the forecasted demand.\n",
    "- *FIRST_WEEK* is the first available week in the data\n",
    "- *LAST_WEEK* is the last available week in the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Forecasting settings\n",
    "N_SPLITS = 1\n",
    "HORIZON = 2\n",
    "GAP = 2\n",
    "FIRST_WEEK = 40\n",
    "LAST_WEEK = 156"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Splitting Time Series Data\n",
    "\n",
    "In all our examples we use `split_train_test()` utility function to split the Orange Juice dataset. Splitting time series data into training and testing has to follow the rule that, in each split, test indices need to be higher than the train indices, and higher than the test indices in the previous split.\n",
    "\n",
    "We wrote the `split_train_test()` function to do just that. Given the parameters listed above, it creates `N_SPLITS` number of training/testing data splits, so that each test split is `HORIZON` weeks long, and the testing period is `GAP` number of weeks away from the training period. The first available week in the data (or the first week we want to start modeling from) is given as `FIRST_WEEK`, and the last week we want to work with is `LAST_WEEK`.\n",
    "\n",
    "For demonstration, this is what the time series split on the Orange Juice dataset looks like, for the parameters listed above.\n",
    "For `HORIZON = 2` and `GAP = 2`, assuming the current week is week `153`, our goal is to forecast the sales in week `155` and `156` using the training data. As you can see, the first forecasting week is `two` weeks away from the current week, as we want to leave time for planning inventory in practice.\n",
    "\n",
    "![Single split](../../../../assets/time_series_split_singleround.jpg)\n",
    "\n",
    "We also refer to splits as rounds, so for `N_SPLITS = 1`, we have single-round forecasting, and for `N_SPLITS > 1`, we have multi-round forecasting."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Single-Round Forecasting\n",
    "\n",
    "Next, we can use `split_train_test()` utility function to split the data in `yx.csv` into training and test sets based on the parameters described above. If we want to do a one-time model training and evaluation, we can split the data using the default settings provided in the function.\n",
    "\n",
    "The data split function will return training data and test data as dataframes. The training data includes `train_df` and `aux_df` with `train_df` containing the historical sales and `aux_df` containing price/promotion information. Here we assume that future price and promotion information up to a certain number of weeks ahead is predetermined and known. The test data is stored in `test_df` which contains the sales of each product.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_df_list, test_df_list, aux_df_list = split_train_test(\n",
    "    data_dir=DATA_DIR,\n",
    "    n_splits=N_SPLITS,\n",
    "    horizon=HORIZON,\n",
    "    gap=GAP,\n",
    "    first_week=FIRST_WEEK,\n",
    "    last_week=LAST_WEEK,\n",
    "    write_csv=True,\n",
    ")\n",
    "\n",
    "# Split returns a list, extract the dataframes from the list\n",
    "train_df = train_df_list[0].reset_index()\n",
    "test_df = test_df_list[0].reset_index()\n",
    "aux_df = aux_df_list[0].reset_index()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>store</th>\n",
       "      <th>brand</th>\n",
       "      <th>week</th>\n",
       "      <th>logmove</th>\n",
       "      <th>constant</th>\n",
       "      <th>price1</th>\n",
       "      <th>price2</th>\n",
       "      <th>price3</th>\n",
       "      <th>price4</th>\n",
       "      <th>price5</th>\n",
       "      <th>price6</th>\n",
       "      <th>price7</th>\n",
       "      <th>price8</th>\n",
       "      <th>price9</th>\n",
       "      <th>price10</th>\n",
       "      <th>price11</th>\n",
       "      <th>deal</th>\n",
       "      <th>feat</th>\n",
       "      <th>profit</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>40</td>\n",
       "      <td>9.018695</td>\n",
       "      <td>1</td>\n",
       "      <td>0.060469</td>\n",
       "      <td>0.060497</td>\n",
       "      <td>0.042031</td>\n",
       "      <td>0.029531</td>\n",
       "      <td>0.049531</td>\n",
       "      <td>0.053021</td>\n",
       "      <td>0.038906</td>\n",
       "      <td>0.041406</td>\n",
       "      <td>0.028906</td>\n",
       "      <td>0.024844</td>\n",
       "      <td>0.038984</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>37.992326</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>46</td>\n",
       "      <td>8.723231</td>\n",
       "      <td>1</td>\n",
       "      <td>0.060469</td>\n",
       "      <td>0.060312</td>\n",
       "      <td>0.045156</td>\n",
       "      <td>0.046719</td>\n",
       "      <td>0.049531</td>\n",
       "      <td>0.047813</td>\n",
       "      <td>0.045781</td>\n",
       "      <td>0.027969</td>\n",
       "      <td>0.042969</td>\n",
       "      <td>0.042031</td>\n",
       "      <td>0.038984</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>30.126667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>47</td>\n",
       "      <td>8.253228</td>\n",
       "      <td>1</td>\n",
       "      <td>0.060469</td>\n",
       "      <td>0.060312</td>\n",
       "      <td>0.045156</td>\n",
       "      <td>0.046719</td>\n",
       "      <td>0.037344</td>\n",
       "      <td>0.053021</td>\n",
       "      <td>0.045781</td>\n",
       "      <td>0.041406</td>\n",
       "      <td>0.048125</td>\n",
       "      <td>0.032656</td>\n",
       "      <td>0.038984</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>30.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>48</td>\n",
       "      <td>8.987197</td>\n",
       "      <td>1</td>\n",
       "      <td>0.060469</td>\n",
       "      <td>0.060312</td>\n",
       "      <td>0.049844</td>\n",
       "      <td>0.037344</td>\n",
       "      <td>0.049531</td>\n",
       "      <td>0.053021</td>\n",
       "      <td>0.045781</td>\n",
       "      <td>0.041406</td>\n",
       "      <td>0.042344</td>\n",
       "      <td>0.032656</td>\n",
       "      <td>0.038984</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>29.950000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>50</td>\n",
       "      <td>9.093357</td>\n",
       "      <td>1</td>\n",
       "      <td>0.060469</td>\n",
       "      <td>0.060312</td>\n",
       "      <td>0.043594</td>\n",
       "      <td>0.031094</td>\n",
       "      <td>0.049531</td>\n",
       "      <td>0.053021</td>\n",
       "      <td>0.046648</td>\n",
       "      <td>0.041406</td>\n",
       "      <td>0.042344</td>\n",
       "      <td>0.032656</td>\n",
       "      <td>0.038203</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>29.920000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>51</td>\n",
       "      <td>8.877382</td>\n",
       "      <td>1</td>\n",
       "      <td>0.060469</td>\n",
       "      <td>0.060312</td>\n",
       "      <td>0.043594</td>\n",
       "      <td>0.046719</td>\n",
       "      <td>0.049531</td>\n",
       "      <td>0.053021</td>\n",
       "      <td>0.046456</td>\n",
       "      <td>0.035781</td>\n",
       "      <td>0.042344</td>\n",
       "      <td>0.029531</td>\n",
       "      <td>0.038203</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>29.920000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>52</td>\n",
       "      <td>9.294682</td>\n",
       "      <td>1</td>\n",
       "      <td>0.051406</td>\n",
       "      <td>0.060312</td>\n",
       "      <td>0.043594</td>\n",
       "      <td>0.046719</td>\n",
       "      <td>0.049531</td>\n",
       "      <td>0.053021</td>\n",
       "      <td>0.047969</td>\n",
       "      <td>0.035781</td>\n",
       "      <td>0.031094</td>\n",
       "      <td>0.029531</td>\n",
       "      <td>0.038984</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>27.125471</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>53</td>\n",
       "      <td>8.954674</td>\n",
       "      <td>1</td>\n",
       "      <td>0.051406</td>\n",
       "      <td>0.060312</td>\n",
       "      <td>0.049844</td>\n",
       "      <td>0.046719</td>\n",
       "      <td>0.034219</td>\n",
       "      <td>0.053021</td>\n",
       "      <td>0.047969</td>\n",
       "      <td>0.035781</td>\n",
       "      <td>0.031094</td>\n",
       "      <td>0.029531</td>\n",
       "      <td>0.038984</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>27.125041</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>54</td>\n",
       "      <td>9.049232</td>\n",
       "      <td>1</td>\n",
       "      <td>0.051406</td>\n",
       "      <td>0.060312</td>\n",
       "      <td>0.049844</td>\n",
       "      <td>0.037344</td>\n",
       "      <td>0.049531</td>\n",
       "      <td>0.053021</td>\n",
       "      <td>0.047969</td>\n",
       "      <td>0.038281</td>\n",
       "      <td>0.048125</td>\n",
       "      <td>0.027969</td>\n",
       "      <td>0.035078</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>27.082481</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>57</td>\n",
       "      <td>8.613230</td>\n",
       "      <td>1</td>\n",
       "      <td>0.051406</td>\n",
       "      <td>0.060312</td>\n",
       "      <td>0.048281</td>\n",
       "      <td>0.046719</td>\n",
       "      <td>0.031094</td>\n",
       "      <td>0.053021</td>\n",
       "      <td>0.031094</td>\n",
       "      <td>0.041406</td>\n",
       "      <td>0.042344</td>\n",
       "      <td>0.042031</td>\n",
       "      <td>0.038984</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "      <td>27.061163</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   store  brand  week   logmove  constant    price1    price2    price3  \\\n",
       "0      2      1    40  9.018695         1  0.060469  0.060497  0.042031   \n",
       "1      2      1    46  8.723231         1  0.060469  0.060312  0.045156   \n",
       "2      2      1    47  8.253228         1  0.060469  0.060312  0.045156   \n",
       "3      2      1    48  8.987197         1  0.060469  0.060312  0.049844   \n",
       "4      2      1    50  9.093357         1  0.060469  0.060312  0.043594   \n",
       "5      2      1    51  8.877382         1  0.060469  0.060312  0.043594   \n",
       "6      2      1    52  9.294682         1  0.051406  0.060312  0.043594   \n",
       "7      2      1    53  8.954674         1  0.051406  0.060312  0.049844   \n",
       "8      2      1    54  9.049232         1  0.051406  0.060312  0.049844   \n",
       "9      2      1    57  8.613230         1  0.051406  0.060312  0.048281   \n",
       "\n",
       "     price4    price5    price6    price7    price8    price9   price10  \\\n",
       "0  0.029531  0.049531  0.053021  0.038906  0.041406  0.028906  0.024844   \n",
       "1  0.046719  0.049531  0.047813  0.045781  0.027969  0.042969  0.042031   \n",
       "2  0.046719  0.037344  0.053021  0.045781  0.041406  0.048125  0.032656   \n",
       "3  0.037344  0.049531  0.053021  0.045781  0.041406  0.042344  0.032656   \n",
       "4  0.031094  0.049531  0.053021  0.046648  0.041406  0.042344  0.032656   \n",
       "5  0.046719  0.049531  0.053021  0.046456  0.035781  0.042344  0.029531   \n",
       "6  0.046719  0.049531  0.053021  0.047969  0.035781  0.031094  0.029531   \n",
       "7  0.046719  0.034219  0.053021  0.047969  0.035781  0.031094  0.029531   \n",
       "8  0.037344  0.049531  0.053021  0.047969  0.038281  0.048125  0.027969   \n",
       "9  0.046719  0.031094  0.053021  0.031094  0.041406  0.042344  0.042031   \n",
       "\n",
       "    price11  deal  feat     profit  \n",
       "0  0.038984     1   0.0  37.992326  \n",
       "1  0.038984     0   0.0  30.126667  \n",
       "2  0.038984     0   0.0  30.000000  \n",
       "3  0.038984     0   0.0  29.950000  \n",
       "4  0.038203     0   0.0  29.920000  \n",
       "5  0.038203     0   0.0  29.920000  \n",
       "6  0.038984     1   0.0  27.125471  \n",
       "7  0.038984     1   0.0  27.125041  \n",
       "8  0.035078     1   0.0  27.082481  \n",
       "9  0.038984     1   0.0  27.061163  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_df.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>store</th>\n",
       "      <th>brand</th>\n",
       "      <th>week</th>\n",
       "      <th>logmove</th>\n",
       "      <th>constant</th>\n",
       "      <th>price1</th>\n",
       "      <th>price2</th>\n",
       "      <th>price3</th>\n",
       "      <th>price4</th>\n",
       "      <th>price5</th>\n",
       "      <th>price6</th>\n",
       "      <th>price7</th>\n",
       "      <th>price8</th>\n",
       "      <th>price9</th>\n",
       "      <th>price10</th>\n",
       "      <th>price11</th>\n",
       "      <th>deal</th>\n",
       "      <th>feat</th>\n",
       "      <th>profit</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>155</td>\n",
       "      <td>9.003070</td>\n",
       "      <td>1</td>\n",
       "      <td>0.042901</td>\n",
       "      <td>0.051979</td>\n",
       "      <td>0.045156</td>\n",
       "      <td>0.031094</td>\n",
       "      <td>0.026406</td>\n",
       "      <td>0.050352</td>\n",
       "      <td>0.041366</td>\n",
       "      <td>0.031094</td>\n",
       "      <td>0.037344</td>\n",
       "      <td>0.018594</td>\n",
       "      <td>0.029453</td>\n",
       "      <td>0</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>26.851024</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>156</td>\n",
       "      <td>9.875911</td>\n",
       "      <td>1</td>\n",
       "      <td>0.038906</td>\n",
       "      <td>0.051979</td>\n",
       "      <td>0.046719</td>\n",
       "      <td>0.031819</td>\n",
       "      <td>0.028320</td>\n",
       "      <td>0.046771</td>\n",
       "      <td>0.040469</td>\n",
       "      <td>0.031094</td>\n",
       "      <td>0.037344</td>\n",
       "      <td>0.032656</td>\n",
       "      <td>0.028047</td>\n",
       "      <td>1</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>22.896316</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>155</td>\n",
       "      <td>8.769041</td>\n",
       "      <td>1</td>\n",
       "      <td>0.042901</td>\n",
       "      <td>0.051979</td>\n",
       "      <td>0.045156</td>\n",
       "      <td>0.031094</td>\n",
       "      <td>0.026406</td>\n",
       "      <td>0.050352</td>\n",
       "      <td>0.041366</td>\n",
       "      <td>0.031094</td>\n",
       "      <td>0.037344</td>\n",
       "      <td>0.018594</td>\n",
       "      <td>0.029453</td>\n",
       "      <td>0</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>29.861343</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>156</td>\n",
       "      <td>8.723231</td>\n",
       "      <td>1</td>\n",
       "      <td>0.038906</td>\n",
       "      <td>0.051979</td>\n",
       "      <td>0.046719</td>\n",
       "      <td>0.031819</td>\n",
       "      <td>0.028320</td>\n",
       "      <td>0.046771</td>\n",
       "      <td>0.040469</td>\n",
       "      <td>0.031094</td>\n",
       "      <td>0.037344</td>\n",
       "      <td>0.032656</td>\n",
       "      <td>0.028047</td>\n",
       "      <td>0</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>29.916250</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>155</td>\n",
       "      <td>8.090709</td>\n",
       "      <td>1</td>\n",
       "      <td>0.042901</td>\n",
       "      <td>0.051979</td>\n",
       "      <td>0.045156</td>\n",
       "      <td>0.031094</td>\n",
       "      <td>0.026406</td>\n",
       "      <td>0.050352</td>\n",
       "      <td>0.041366</td>\n",
       "      <td>0.031094</td>\n",
       "      <td>0.037344</td>\n",
       "      <td>0.018594</td>\n",
       "      <td>0.029453</td>\n",
       "      <td>1</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>36.118103</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>156</td>\n",
       "      <td>7.336937</td>\n",
       "      <td>1</td>\n",
       "      <td>0.038906</td>\n",
       "      <td>0.051979</td>\n",
       "      <td>0.046719</td>\n",
       "      <td>0.031819</td>\n",
       "      <td>0.028320</td>\n",
       "      <td>0.046771</td>\n",
       "      <td>0.040469</td>\n",
       "      <td>0.031094</td>\n",
       "      <td>0.037344</td>\n",
       "      <td>0.032656</td>\n",
       "      <td>0.028047</td>\n",
       "      <td>0</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>39.047500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>2</td>\n",
       "      <td>4</td>\n",
       "      <td>155</td>\n",
       "      <td>9.869310</td>\n",
       "      <td>1</td>\n",
       "      <td>0.042901</td>\n",
       "      <td>0.051979</td>\n",
       "      <td>0.045156</td>\n",
       "      <td>0.031094</td>\n",
       "      <td>0.026406</td>\n",
       "      <td>0.050352</td>\n",
       "      <td>0.041366</td>\n",
       "      <td>0.031094</td>\n",
       "      <td>0.037344</td>\n",
       "      <td>0.018594</td>\n",
       "      <td>0.029453</td>\n",
       "      <td>1</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>23.502053</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>2</td>\n",
       "      <td>4</td>\n",
       "      <td>156</td>\n",
       "      <td>9.135617</td>\n",
       "      <td>1</td>\n",
       "      <td>0.038906</td>\n",
       "      <td>0.051979</td>\n",
       "      <td>0.046719</td>\n",
       "      <td>0.031819</td>\n",
       "      <td>0.028320</td>\n",
       "      <td>0.046771</td>\n",
       "      <td>0.040469</td>\n",
       "      <td>0.031094</td>\n",
       "      <td>0.037344</td>\n",
       "      <td>0.032656</td>\n",
       "      <td>0.028047</td>\n",
       "      <td>1</td>\n",
       "      <td>0.97379</td>\n",
       "      <td>26.420000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>2</td>\n",
       "      <td>5</td>\n",
       "      <td>155</td>\n",
       "      <td>9.852615</td>\n",
       "      <td>1</td>\n",
       "      <td>0.042901</td>\n",
       "      <td>0.051979</td>\n",
       "      <td>0.045156</td>\n",
       "      <td>0.031094</td>\n",
       "      <td>0.026406</td>\n",
       "      <td>0.050352</td>\n",
       "      <td>0.041366</td>\n",
       "      <td>0.031094</td>\n",
       "      <td>0.037344</td>\n",
       "      <td>0.018594</td>\n",
       "      <td>0.029453</td>\n",
       "      <td>1</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>19.999910</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>2</td>\n",
       "      <td>5</td>\n",
       "      <td>156</td>\n",
       "      <td>9.357380</td>\n",
       "      <td>1</td>\n",
       "      <td>0.038906</td>\n",
       "      <td>0.051979</td>\n",
       "      <td>0.046719</td>\n",
       "      <td>0.031819</td>\n",
       "      <td>0.028320</td>\n",
       "      <td>0.046771</td>\n",
       "      <td>0.040469</td>\n",
       "      <td>0.031094</td>\n",
       "      <td>0.037344</td>\n",
       "      <td>0.032656</td>\n",
       "      <td>0.028047</td>\n",
       "      <td>1</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>24.212895</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   store  brand  week   logmove  constant    price1    price2    price3  \\\n",
       "0      2      1   155  9.003070         1  0.042901  0.051979  0.045156   \n",
       "1      2      1   156  9.875911         1  0.038906  0.051979  0.046719   \n",
       "2      2      2   155  8.769041         1  0.042901  0.051979  0.045156   \n",
       "3      2      2   156  8.723231         1  0.038906  0.051979  0.046719   \n",
       "4      2      3   155  8.090709         1  0.042901  0.051979  0.045156   \n",
       "5      2      3   156  7.336937         1  0.038906  0.051979  0.046719   \n",
       "6      2      4   155  9.869310         1  0.042901  0.051979  0.045156   \n",
       "7      2      4   156  9.135617         1  0.038906  0.051979  0.046719   \n",
       "8      2      5   155  9.852615         1  0.042901  0.051979  0.045156   \n",
       "9      2      5   156  9.357380         1  0.038906  0.051979  0.046719   \n",
       "\n",
       "     price4    price5    price6    price7    price8    price9   price10  \\\n",
       "0  0.031094  0.026406  0.050352  0.041366  0.031094  0.037344  0.018594   \n",
       "1  0.031819  0.028320  0.046771  0.040469  0.031094  0.037344  0.032656   \n",
       "2  0.031094  0.026406  0.050352  0.041366  0.031094  0.037344  0.018594   \n",
       "3  0.031819  0.028320  0.046771  0.040469  0.031094  0.037344  0.032656   \n",
       "4  0.031094  0.026406  0.050352  0.041366  0.031094  0.037344  0.018594   \n",
       "5  0.031819  0.028320  0.046771  0.040469  0.031094  0.037344  0.032656   \n",
       "6  0.031094  0.026406  0.050352  0.041366  0.031094  0.037344  0.018594   \n",
       "7  0.031819  0.028320  0.046771  0.040469  0.031094  0.037344  0.032656   \n",
       "8  0.031094  0.026406  0.050352  0.041366  0.031094  0.037344  0.018594   \n",
       "9  0.031819  0.028320  0.046771  0.040469  0.031094  0.037344  0.032656   \n",
       "\n",
       "    price11  deal     feat     profit  \n",
       "0  0.029453     0  0.00000  26.851024  \n",
       "1  0.028047     1  1.00000  22.896316  \n",
       "2  0.029453     0  0.00000  29.861343  \n",
       "3  0.028047     0  0.00000  29.916250  \n",
       "4  0.029453     1  1.00000  36.118103  \n",
       "5  0.028047     0  0.00000  39.047500  \n",
       "6  0.029453     1  1.00000  23.502053  \n",
       "7  0.028047     1  0.97379  26.420000  \n",
       "8  0.029453     1  0.00000  19.999910  \n",
       "9  0.028047     1  0.00000  24.212895  "
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_df.head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>store</th>\n",
       "      <th>brand</th>\n",
       "      <th>week</th>\n",
       "      <th>price1</th>\n",
       "      <th>price2</th>\n",
       "      <th>price3</th>\n",
       "      <th>price4</th>\n",
       "      <th>price5</th>\n",
       "      <th>price6</th>\n",
       "      <th>price7</th>\n",
       "      <th>price8</th>\n",
       "      <th>price9</th>\n",
       "      <th>price10</th>\n",
       "      <th>price11</th>\n",
       "      <th>deal</th>\n",
       "      <th>feat</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>40</td>\n",
       "      <td>0.060469</td>\n",
       "      <td>0.060497</td>\n",
       "      <td>0.042031</td>\n",
       "      <td>0.029531</td>\n",
       "      <td>0.049531</td>\n",
       "      <td>0.053021</td>\n",
       "      <td>0.038906</td>\n",
       "      <td>0.041406</td>\n",
       "      <td>0.028906</td>\n",
       "      <td>0.024844</td>\n",
       "      <td>0.038984</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>46</td>\n",
       "      <td>0.060469</td>\n",
       "      <td>0.060312</td>\n",
       "      <td>0.045156</td>\n",
       "      <td>0.046719</td>\n",
       "      <td>0.049531</td>\n",
       "      <td>0.047813</td>\n",
       "      <td>0.045781</td>\n",
       "      <td>0.027969</td>\n",
       "      <td>0.042969</td>\n",
       "      <td>0.042031</td>\n",
       "      <td>0.038984</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>47</td>\n",
       "      <td>0.060469</td>\n",
       "      <td>0.060312</td>\n",
       "      <td>0.045156</td>\n",
       "      <td>0.046719</td>\n",
       "      <td>0.037344</td>\n",
       "      <td>0.053021</td>\n",
       "      <td>0.045781</td>\n",
       "      <td>0.041406</td>\n",
       "      <td>0.048125</td>\n",
       "      <td>0.032656</td>\n",
       "      <td>0.038984</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>48</td>\n",
       "      <td>0.060469</td>\n",
       "      <td>0.060312</td>\n",
       "      <td>0.049844</td>\n",
       "      <td>0.037344</td>\n",
       "      <td>0.049531</td>\n",
       "      <td>0.053021</td>\n",
       "      <td>0.045781</td>\n",
       "      <td>0.041406</td>\n",
       "      <td>0.042344</td>\n",
       "      <td>0.032656</td>\n",
       "      <td>0.038984</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>50</td>\n",
       "      <td>0.060469</td>\n",
       "      <td>0.060312</td>\n",
       "      <td>0.043594</td>\n",
       "      <td>0.031094</td>\n",
       "      <td>0.049531</td>\n",
       "      <td>0.053021</td>\n",
       "      <td>0.046648</td>\n",
       "      <td>0.041406</td>\n",
       "      <td>0.042344</td>\n",
       "      <td>0.032656</td>\n",
       "      <td>0.038203</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>51</td>\n",
       "      <td>0.060469</td>\n",
       "      <td>0.060312</td>\n",
       "      <td>0.043594</td>\n",
       "      <td>0.046719</td>\n",
       "      <td>0.049531</td>\n",
       "      <td>0.053021</td>\n",
       "      <td>0.046456</td>\n",
       "      <td>0.035781</td>\n",
       "      <td>0.042344</td>\n",
       "      <td>0.029531</td>\n",
       "      <td>0.038203</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>52</td>\n",
       "      <td>0.051406</td>\n",
       "      <td>0.060312</td>\n",
       "      <td>0.043594</td>\n",
       "      <td>0.046719</td>\n",
       "      <td>0.049531</td>\n",
       "      <td>0.053021</td>\n",
       "      <td>0.047969</td>\n",
       "      <td>0.035781</td>\n",
       "      <td>0.031094</td>\n",
       "      <td>0.029531</td>\n",
       "      <td>0.038984</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>53</td>\n",
       "      <td>0.051406</td>\n",
       "      <td>0.060312</td>\n",
       "      <td>0.049844</td>\n",
       "      <td>0.046719</td>\n",
       "      <td>0.034219</td>\n",
       "      <td>0.053021</td>\n",
       "      <td>0.047969</td>\n",
       "      <td>0.035781</td>\n",
       "      <td>0.031094</td>\n",
       "      <td>0.029531</td>\n",
       "      <td>0.038984</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>54</td>\n",
       "      <td>0.051406</td>\n",
       "      <td>0.060312</td>\n",
       "      <td>0.049844</td>\n",
       "      <td>0.037344</td>\n",
       "      <td>0.049531</td>\n",
       "      <td>0.053021</td>\n",
       "      <td>0.047969</td>\n",
       "      <td>0.038281</td>\n",
       "      <td>0.048125</td>\n",
       "      <td>0.027969</td>\n",
       "      <td>0.035078</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>57</td>\n",
       "      <td>0.051406</td>\n",
       "      <td>0.060312</td>\n",
       "      <td>0.048281</td>\n",
       "      <td>0.046719</td>\n",
       "      <td>0.031094</td>\n",
       "      <td>0.053021</td>\n",
       "      <td>0.031094</td>\n",
       "      <td>0.041406</td>\n",
       "      <td>0.042344</td>\n",
       "      <td>0.042031</td>\n",
       "      <td>0.038984</td>\n",
       "      <td>1</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   store  brand  week    price1    price2    price3    price4    price5  \\\n",
       "0      2      1    40  0.060469  0.060497  0.042031  0.029531  0.049531   \n",
       "1      2      1    46  0.060469  0.060312  0.045156  0.046719  0.049531   \n",
       "2      2      1    47  0.060469  0.060312  0.045156  0.046719  0.037344   \n",
       "3      2      1    48  0.060469  0.060312  0.049844  0.037344  0.049531   \n",
       "4      2      1    50  0.060469  0.060312  0.043594  0.031094  0.049531   \n",
       "5      2      1    51  0.060469  0.060312  0.043594  0.046719  0.049531   \n",
       "6      2      1    52  0.051406  0.060312  0.043594  0.046719  0.049531   \n",
       "7      2      1    53  0.051406  0.060312  0.049844  0.046719  0.034219   \n",
       "8      2      1    54  0.051406  0.060312  0.049844  0.037344  0.049531   \n",
       "9      2      1    57  0.051406  0.060312  0.048281  0.046719  0.031094   \n",
       "\n",
       "     price6    price7    price8    price9   price10   price11  deal  feat  \n",
       "0  0.053021  0.038906  0.041406  0.028906  0.024844  0.038984     1   0.0  \n",
       "1  0.047813  0.045781  0.027969  0.042969  0.042031  0.038984     0   0.0  \n",
       "2  0.053021  0.045781  0.041406  0.048125  0.032656  0.038984     0   0.0  \n",
       "3  0.053021  0.045781  0.041406  0.042344  0.032656  0.038984     0   0.0  \n",
       "4  0.053021  0.046648  0.041406  0.042344  0.032656  0.038203     0   0.0  \n",
       "5  0.053021  0.046456  0.035781  0.042344  0.029531  0.038203     0   0.0  \n",
       "6  0.053021  0.047969  0.035781  0.031094  0.029531  0.038984     1   0.0  \n",
       "7  0.053021  0.047969  0.035781  0.031094  0.029531  0.038984     1   0.0  \n",
       "8  0.053021  0.047969  0.038281  0.048125  0.027969  0.035078     1   0.0  \n",
       "9  0.053021  0.031094  0.041406  0.042344  0.042031  0.038984     1   0.0  "
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "aux_df.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Multi-Round Forecasting\n",
    "\n",
    "To create training data and test data for multi-round forecasting, we use the same function passing a number greater than `1` to `n_splits` parameter. Note that the forecasting period we generate in each test round are **non-overlapping**. This allows us to evaluate the forecasting model on multiple rounds of data, and get a more robust estimate of our model's performance.\n",
    "\n",
    "For demonstration, this is what the time series splits would look like for `N_SPLITS = 5`, and using other settings as above:\n",
    "\n",
    "![Multi split](../../../../assets/time_series_split_multiround.jpg)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's now generate `10` rounds of training/testing data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "lines_to_next_cell": 2
   },
   "outputs": [],
   "source": [
    "## Generate 10 splits (rounds) of data\n",
    "N_SPLITS = 10\n",
    "\n",
    "train_df_list, test_df_list, aux_df_list = split_train_test(\n",
    "    DATA_DIR, n_splits=N_SPLITS, horizon=HORIZON, gap=GAP, first_week=FIRST_WEEK, last_week=LAST_WEEK, write_csv=True,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The above cell wil generate the following splits.\n",
    "\n",
    "| **Round** | **Train period in weeks** | **Test period in weeks** |\n",
    "| -------- | -------------------------- | ----------------------- |\n",
    "| 1 | 40 - 135 | 137 - 138 |\n",
    "| 2 | 40 - 137 | 139 - 140 |\n",
    "| 3 | 40 - 139 | 141 - 142 |\n",
    "| 4 | 40 - 141 | 143 - 144 |\n",
    "| 5 | 40 - 143 | 145 - 146 |\n",
    "| 6 | 40 - 145 | 147 - 148 |\n",
    "| 7 | 40 - 147 | 149 - 150 |\n",
    "| 8 | 40 - 149 | 151 - 152 |\n",
    "| 9 | 40 - 151 | 153 - 154 |\n",
    "| 10 | 40 - 153 | 155 - 156 |\n",
    "\n",
    "The gap of one week between training period and test period allows store managers to prepare the stock based on the forecasted demand. Besides, we assume that the information about the price, deal, and advertisement up until the forecast period end week is available at each round."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ROUND 1\n",
      "--------\n",
      "Training\n",
      "    Data shape: (84183, 18)\n",
      "    Week range: 40-135\n",
      "Testing\n",
      "    Data shape: (1826, 18)\n",
      "    Week range: 137-138\n",
      "Auxiliary\n",
      "    Data shape: (86911, 15)\n",
      "    Week range: 40-138\n",
      "\n",
      "ROUND 2\n",
      "--------\n",
      "Training\n",
      "    Data shape: (85998, 18)\n",
      "    Week range: 40-137\n",
      "Testing\n",
      "    Data shape: (1793, 18)\n",
      "    Week range: 139-140\n",
      "Auxiliary\n",
      "    Data shape: (88704, 15)\n",
      "    Week range: 40-140\n",
      "\n",
      "ROUND 3\n",
      "--------\n",
      "Training\n",
      "    Data shape: (87802, 18)\n",
      "    Week range: 40-139\n",
      "Testing\n",
      "    Data shape: (1771, 18)\n",
      "    Week range: 141-142\n",
      "Auxiliary\n",
      "    Data shape: (90475, 15)\n",
      "    Week range: 40-142\n",
      "\n",
      "ROUND 4\n",
      "--------\n",
      "Training\n",
      "    Data shape: (89617, 18)\n",
      "    Week range: 40-141\n",
      "Testing\n",
      "    Data shape: (1749, 18)\n",
      "    Week range: 143-144\n",
      "Auxiliary\n",
      "    Data shape: (92224, 15)\n",
      "    Week range: 40-144\n",
      "\n",
      "ROUND 5\n",
      "--------\n",
      "Training\n",
      "    Data shape: (91333, 18)\n",
      "    Week range: 40-143\n",
      "Testing\n",
      "    Data shape: (1727, 18)\n",
      "    Week range: 145-146\n",
      "Auxiliary\n",
      "    Data shape: (93951, 15)\n",
      "    Week range: 40-146\n",
      "\n",
      "ROUND 6\n",
      "--------\n",
      "Training\n",
      "    Data shape: (93071, 18)\n",
      "    Week range: 40-145\n",
      "Testing\n",
      "    Data shape: (1749, 18)\n",
      "    Week range: 147-148\n",
      "Auxiliary\n",
      "    Data shape: (95700, 15)\n",
      "    Week range: 40-148\n",
      "\n",
      "ROUND 7\n",
      "--------\n",
      "Training\n",
      "    Data shape: (94842, 18)\n",
      "    Week range: 40-147\n",
      "Testing\n",
      "    Data shape: (1771, 18)\n",
      "    Week range: 149-150\n",
      "Auxiliary\n",
      "    Data shape: (97471, 15)\n",
      "    Week range: 40-150\n",
      "\n",
      "ROUND 8\n",
      "--------\n",
      "Training\n",
      "    Data shape: (96591, 18)\n",
      "    Week range: 40-149\n",
      "Testing\n",
      "    Data shape: (1738, 18)\n",
      "    Week range: 151-152\n",
      "Auxiliary\n",
      "    Data shape: (99209, 15)\n",
      "    Week range: 40-152\n",
      "\n",
      "ROUND 9\n",
      "--------\n",
      "Training\n",
      "    Data shape: (98340, 18)\n",
      "    Week range: 40-151\n",
      "Testing\n",
      "    Data shape: (1705, 18)\n",
      "    Week range: 153-154\n",
      "Auxiliary\n",
      "    Data shape: (100914, 15)\n",
      "    Week range: 40-154\n",
      "\n",
      "ROUND 10\n",
      "--------\n",
      "Training\n",
      "    Data shape: (100056, 18)\n",
      "    Week range: 40-153\n",
      "Testing\n",
      "    Data shape: (1705, 18)\n",
      "    Week range: 155-156\n",
      "Auxiliary\n",
      "    Data shape: (102619, 15)\n",
      "    Week range: 40-156\n",
      "\n"
     ]
    }
   ],
   "source": [
    "for i in range(len(train_df_list)):\n",
    "    train_df = train_df_list[i]\n",
    "    test_df = test_df_list[i]\n",
    "    aux_df = aux_df_list[i]\n",
    "    print(f\"ROUND {i+1}\")\n",
    "    print(\"--------\")\n",
    "    print(\"Training\")\n",
    "    print(f\"    Data shape: {train_df.shape}\")\n",
    "    print(f\"    Week range: {min(train_df['week'])}-{max(train_df['week'])}\")\n",
    "    print(\"Testing\")\n",
    "    print(f\"    Data shape: {test_df.shape}\")\n",
    "    print(f\"    Week range: {min(test_df['week'])}-{max(test_df['week'])}\")\n",
    "    print(\"Auxiliary\")\n",
    "    print(f\"    Data shape: {aux_df.shape}\")\n",
    "    print(f\"    Week range: {min(aux_df['week'])}-{max(aux_df['week'])}\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Additional Reading\n",
    "\n",
    "\\[1\\] Christoph GBergmeir, Rob J. Hyndman, and Bonsoo Koo. 2018. A Note on the Validity of Cross-Validation for Evaluating Autoregressive Time Series Prediction. Computational Statistics & Data Analysis. 120, pp. 70-83.<br>\n",
    "\\[2\\] How To Backtest Machine Learning Models for Time Series Forecasting: https://machinelearningmastery.com/backtest-machine-learning-models-time-series-forecasting/Parameters.rst <br>\n",
    "\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "forecasting_env",
   "language": "python",
   "name": "forecasting_env"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}