Assignment 8.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# ``Assignment``: Predicting Daily Temperatures using Artificial Neural Networks\n",
    "\n",
    "## ``Objective``:\n",
    "To design and implement an artificial neural network using PyTorch to predict daily temperatures based on historical weather data. This project will involve data preprocessing, building, and training an ANN model, and evaluating its performance.\n",
    "\n",
    "## ``Dataset``:\n",
    "We will use the **Daily minimum temperatures in Melbourne dataset** from the `UCI Machine Learning Repository` or Kaggle. This dataset contains daily temperature observations for Melbourne, Australia, over a period of 10 years, 1981-1990.\n",
    "\n",
    "## ``Importance and Analysis``\n",
    "\n",
    "### Nature of the Project\n",
    "This project focuses on time-series prediction, which is a critical task in weather data science. Weather data is naturally sequential; the temperature on one day is often influenced by temperatures on previous days. Predicting future temperatures based on past observations can help in planning and decision-making in various sectors like agriculture, energy, and public safety.\n",
    "\n",
    "### Importance of Sequence Data\n",
    "**Why Use Sequence Data?**\n",
    "- In time-series prediction, sequences capture temporal dependencies in the data. By considering the temperatures of the past few days, the model can learn patterns and trends that are useful for predicting future temperatures. This sequential information is crucial for accurate predictions.\n",
    "\n",
    "### Relevance of Sequences in Weather Data Science\n",
    "Weather patterns are inherently sequential. For instance, a cold front or heatwave affects temperatures over several days. By using sequences, we allow the model to understand these patterns and make more informed predictions.\n",
    "\n",
    "### Handling Sequences in ANNs\n",
    "In this project, we will create sequences of temperature data to feed into our neural network. Here's how we'll handle sequences:\n",
    "\n",
    "1. **Sequence Length**: We'll use the past 7 days to predict the next day's temperature. This is our sequence length.\n",
    "2. **Input Shape**: Each input to the model will be a sequence of 7 days of temperature data.\n",
    "3. **Output Shape**: The output will be a single value, representing the predicted temperature for the next day."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## ``Tasks``:\n",
    "\n",
    "### Part 1: Data Loading and Preprocessing\n",
    "\n",
    "## Step 1: Download the Dataset\n",
    "\n",
    "Download the **Daily minimum temperatures in Melbourne dataset** from the following link:\n",
    "\n",
    "[Kaggle: Daily Minimum Temperatures in Melbourne Dataset](https://www.kaggle.com/datasets/paulbrabban/daily-minimum-temperatures-in-melbourne)\n",
    "\n",
    "1. Go to the provided link.\n",
    "2. Click on the \"Download\" button to download the dataset as a CSV file.\n",
    "3. Extract the CSV file from the downloaded ZIP file if it is compressed.\n",
    "4. Rename the CSV file to `daily-min-temperatures.csv` if it has a different name.\n",
    "\n",
    "#### 1. Load the Weather Dataset\n",
    "Use `pandas` to load the dataset from a CSV file.\n",
    "\n",
    "```python\n",
    "import pandas as pd\n",
    "\n",
    "# Load the dataset\n",
    "df = pd.read_csv('daily-min-temperatures.csv')\n",
    "\n",
    "# Display the first few rows of the dataframe\n",
    "print(df.head()) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2. Data Cleaning\n",
    "\n",
    "- Look at the data, you may need to remove unecessary rows (may be last useless row), additionally you may also need to prepare the dataframe properly.\n",
    "- Handle missing values appropriately (e.g., imputation, removal). Check for NaN and infinities, if present use forward fill method to replace the NaN.\n",
    "\n",
    "```python\n",
    "# Fill missing values with the forward fill method\n",
    "df.fillna(method='ffill', inplace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3. Feature Engineering \n",
    "( Although you will not be using this erived feature in the actual modelling part, this section is just to let you know the concept of derived features)\n",
    "- Create new features based on the existing data (e.g., rolling averages, time-based features).\n",
    "\n",
    "#### Why Feature Engineering?\n",
    "Feature engineering helps in creating additional relevant features from the existing data, which can provide the model with more information and potentially improve its performance. For example, using rolling averages can help smooth out the data and highlight trends.\n",
    "\n",
    "```python\n",
    "# Create rolling average features (you should try to learn more about rolling average and its importance in weather)\n",
    "df['Temp_rolling_mean'] = df['Temp'].rolling(window=7).mean()\n",
    "\n",
    "# Drop rows with NaN values generated by rolling mean\n",
    "df.dropna(inplace=True)\n",
    "\n",
    "# Display the first few rows of the dataframe with the new feature\n",
    "print(df.head())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 4. Data Normalization\n",
    "Normalize the temperature values to have values between 0 and 1.\n",
    "\n",
    "#### Why Normalize?\n",
    "Normalization scales the data to a common range, which can help improve the convergence rate during training and lead to better performance.\n",
    "\n",
    "```python\n",
    "from sklearn.preprocessing import MinMaxScaler\n",
    "\n",
    "# Initialize the scaler\n",
    "scaler = MinMaxScaler()\n",
    "\n",
    "# Normalize the temperature values\n",
    "df['Temp'] = scaler.fit_transform(df[['Temp']])\n",
    "df['Temp_rolling_mean'] = scaler.fit_transform(df[['Temp_rolling_mean']])\n",
    "\n",
    "# Display the first few rows of the dataframe after normalization\n",
    "print(df.head())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 5. Sequence Generation\n",
    "Create sequences of data for time-series prediction (e.g., use the past 7 days to predict the next day's temperature). Only use temperature for sequence generation, donot use rolling average.\n",
    "\n",
    "#### Why Sequence?\n",
    "Time-series data is inherently sequential, and using sequences allows the model to learn temporal dependencies. For instance, knowing the temperatures of the past 7 days helps predict the temperature for the next day.\n",
    "\n",
    "```python\n",
    "import numpy as np\n",
    "\n",
    "def create_sequences(data, seq_length):\n",
    "    sequences = []\n",
    "    labels = []\n",
    "    for i in range(len(data) - seq_length):\n",
    "        seq = data[i:i + seq_length]\n",
    "        label = data[i + seq_length]\n",
    "        sequences.append(seq)\n",
    "        labels.append(label)\n",
    "    return np.array(sequences), np.array(labels)\n",
    "\n",
    "# Define the sequence length (let say 7)\n",
    "seq_length = 7\n",
    "\n",
    "# Create sequences of data\n",
    "X, y = create_sequences(df['Temp'].values, seq_length)\n",
    "\n",
    "# Display the shape of the generated sequences\n",
    "print(f\"Shape of X: {X.shape}\")\n",
    "print(f\"Shape of y: {y.shape}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Part 2: Building the Neural Network\n",
    "Define the neural network architecture, implement the training loop, and train the model using the training data.\n",
    "\n",
    "#### 1. Define the Neural Network Architecture:\n",
    "* Input layer: Number of input neurons equal to the number of days used for prediction (e.g., 7).\n",
    "* Hidden layers: Two hidden layers, each with 64 neurons and ReLU activation.\n",
    "* Output layer: 1 neuron (for the predicted temperature)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Part 3: Training the Neural Network\n",
    "#### 1. Define the Loss Function and Optimizer:\n",
    "* Use Mean Squared Error (MSE) for the loss function.\n",
    "* Use Adam for the optimizer.\n",
    "\n",
    "#### 2. Train the Network:\n",
    "* Implement the training loop. Split your data into train and test set, and train on train set.\n",
    "* Track the loss during training.\n",
    "\n",
    "```python\n",
    "\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "# Split the data into training and testing sets\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, shuffle=False)\n",
    "\n",
    "# Convert the data to PyTorch tensors\n",
    "X_train = torch.tensor(X, dtype=torch.float32)\n",
    "y_train = torch.tensor(y, dtype=torch.float32).view(-1, 1)\n",
    "X_test = torch.tensor(X_test, dtype=torch.float32)\n",
    "y_test = torch.tensor(y_test, dtype=torch.float32).view(-1, 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Part 4: Evaluating the Model\n",
    "#### 1. Evaluate the Model on Test Data:\n",
    "Calculate the Root Mean Squared Error (RMSE) of the model on the test dataset.\n",
    "Plot the actual vs. predicted temperatures.\n",
    "\n",
    "```python\n",
    "# Evaluate the model\n",
    "model.eval()\n",
    "with torch.no_grad():\n",
    "    predictions = model(X_test)\n",
    "    rmse = torch.sqrt(criterion(predictions, y_test))\n",
    "    print(f\"Test RMSE: {rmse.item()}\")\n",
    "\n",
    "# Plot actual vs. predicted temperatures\n",
    "plt.figure(figsize=(10, 6))\n",
    "plt.plot(y_test.numpy(), label='Actual')\n",
    "plt.plot(predictions.numpy(), label='Predicted')\n",
    "plt.legend()\n",
    "plt.xlabel('Day')\n",
    "plt.ylabel('Normalized Temperature')\n",
    "plt.title('Actual vs. Predicted Temperatures')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Part 5: Making Predictions and Analysis\n",
    "#### 1. Make Predictions on New Data:\n",
    "Use the trained model to make predictions on new data.\n",
    "Display some example predictions with the corresponding actual temperatures.\n",
    "\n",
    "```python\n",
    "# Generate new sequences for predictions (e.g., the last 7 days in the dataset)\n",
    "new_data = df['Temp'].values[-seq_length:]\n",
    "new_data = torch.tensor(new_data, dtype=torch.float32).view(1, -1)\n",
    "\n",
    "# Make predictions\n",
    "model.eval()\n",
    "with torch.no_grad():\n",
    "    new_predictions = model(new_data)\n",
    "\n",
    "print(f\"Predicted temperature for the next day: {new_predictions.item()}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2. Analyze Model Performance:\n",
    "Discuss potential improvements and further steps. (This section is not mandatory but recommended)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# ``Special section``"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Special Section 1: Analysis of Model Performance\n",
    "\n",
    "- The RMSE on the test data provides an indication of the model's performance. \n",
    "- Compare the actual vs. predicted temperatures plot to visually assess the model's predictions.\n",
    "- Potential improvements could include tuning the hyperparameters, adding more features, or using a different model architecture.\n",
    "- Further steps could involve experimenting with different sequence lengths or using more advanced techniques like LSTM networks for time-series prediction."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Special Section 2: Preparation of Train and Test Data\n",
    "\n",
    "### Understanding Train and Test Data Preparation\n",
    "\n",
    "In time-series prediction, the preparation of train and test data is different from typical supervised learning tasks. Here, if the input `X` is the temperature data for 7 days, the corresponding ground truth `y` is the temperature on the 8th day. This approach helps the model learn the temporal dependencies in the data.\n",
    "\n",
    "### Why is This Important?\n",
    "\n",
    "Time-series data has a natural order, and it is crucial to maintain this order when splitting the data into training and testing sets. Shuffling the data would break the temporal relationships, leading to poor model performance.\n",
    "\n",
    "### Step-by-Step Process\n",
    "\n",
    "1. **Create Sequences**: Generate sequences of data where each sequence contains data for 7 consecutive days.\n",
    "2. **Split the Data**: Split the sequences into training and testing sets without shuffling to maintain the temporal order.\n",
    "3. **Prepare Tensors**: Convert the sequences into PyTorch tensors for model training.\n",
    "\n",
    "### Code Snippet for Data Preparation\n",
    "\n",
    "```python\n",
    "import numpy as np\n",
    "from sklearn.model_selection import train_test_split\n",
    "import torch\n",
    "\n",
    "# Define the function to create sequences\n",
    "def create_sequences(data, seq_length):\n",
    "    sequences = []\n",
    "    labels = []\n",
    "    for i in range(len(data) - seq_length):\n",
    "        seq = data[i:i + seq_length]\n",
    "        label = data[i + seq_length]\n",
    "        sequences.append(seq)\n",
    "        labels.append(label)\n",
    "    return np.array(sequences), np.array(labels)\n",
    "\n",
    "# Define the sequence length\n",
    "seq_length = 7\n",
    "\n",
    "# Create sequences of data\n",
    "X, y = create_sequences(df['Temp'].values, seq_length)\n",
    "\n",
    "# Split the data into training and testing sets without shuffling\n",
    "train_size = int(len(X) * 0.8)\n",
    "X_train, X_test = X[:train_size], X[train_size:]\n",
    "y_train, y_test = y[:train_size], y[train_size:]\n",
    "\n",
    "# Convert the data to PyTorch tensors\n",
    "X_train = torch.tensor(X_train, dtype=torch.float32)\n",
    "X_test = torch.tensor(X_test, dtype=torch.float32)\n",
    "y_train = torch.tensor(y_train, dtype=torch.float32).view(-1, 1)\n",
    "y_test = torch.tensor(y_test, dtype=torch.float32).view(-1, 1)\n",
    "\n",
    "# Display the shape of the train and test sets\n",
    "print(f\"Shape of X_train: {X_train.shape}\")\n",
    "print(f\"Shape of y_train: {y_train.shape}\")\n",
    "print(f\"Shape of X_test: {X_test.shape}\")\n",
    "print(f\"Shape of y_test: {y_test.shape}\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "pymc_env",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3.11.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}