11.Pytorch Custom Dataset and Data loader.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "source": [
    "### Data Loader and Custom Dataset\r\n",
    "---\r\n",
    "\r\n",
    "Dataset : MNIST\r\n",
    "* Train Data : [Train](https://pjreddie.com/media/files/mnist_train.csv) - 60000 Training Samples\r\n",
    "* Test : [Test](https://pjreddie.com/media/files/mnist_test.csv) - 10000 Testing Samples\r\n",
    "\r\n",
    "\r\n",
    "### The Dataset Class\r\n",
    "---\r\n",
    "The Dataset class has three important class functions:\r\n",
    "* `__init__()`: as usual, the starting point where we will initialize everything that we use in the class.\r\n",
    "* `__len__()`: this returns the length of the dataset. Simply this will return the number of samples in the dataset.\r\n",
    "* `__getitem__()`: this function returns a sample from the dataset when we provide an index value to it.  "
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### **1.Small Example Dataset Class**"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "source": [
    "'''\r\n",
    "We can do amazing things with PyTorch Dataset class. We need to ensure that we are overriding two \r\n",
    "of it's functions, \r\n",
    "`__len__()`: returns the size of the dataset, that is, total number of samples.\r\n",
    "`__getitem__()`: when given an index, returns the data sample correspoding to that index.\r\n",
    "'''\r\n",
    "\r\n",
    "import numpy as np\r\n",
    "from torch.utils.data import Dataset"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "Define the Class ExampleDataset"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "source": [
    "class ExampleDataset(Dataset):\r\n",
    "    def __init__(self, data):\r\n",
    "        self.data = data\r\n",
    "        \r\n",
    "    def __len__(self):\r\n",
    "        return len(self.data)\r\n",
    "    \r\n",
    "    def __getitem__(self, index):\r\n",
    "        return self.data[index]"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "Running the dataset"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "source": [
    "sample_data = np.arange(0,10)\r\n",
    "print('the whole data:', sample_data)\r\n",
    "\r\n",
    "dataset = ExampleDataset(sample_data)\r\n",
    "print('Number of samples in the data:', len(dataset))\r\n",
    "\r\n",
    "print(dataset[0])\r\n",
    "print(dataset[0:5])"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "the whole data: [0 1 2 3 4 5 6 7 8 9]\n",
      "Number of samples in the data: 10\n",
      "0\n",
      "[0 1 2 3 4]\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### **Creating Datasets and DataLoaders from CSV Files**\r\n",
    "\r\n",
    "\r\n",
    "### **1. Load Libraries**"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "source": [
    "# torch loaded...!!!\r\n",
    "import torch\r\n",
    "import torch.nn as nn\r\n",
    "import torch.nn.functional as F\r\n",
    "import torch.optim as optim\r\n",
    "from torch.utils.data import DataLoader\r\n",
    "from torch.utils.data import Dataset\r\n",
    "\r\n",
    "# torchvision loaded...!!!\r\n",
    "import torchvision\r\n",
    "import torchvision.transforms as transforms\r\n",
    "from torchvision.utils import save_image\r\n",
    "from torchsummary import summary\r\n",
    "\r\n",
    "# other Libraries loaded...!!!\r\n",
    "import os\r\n",
    "import pandas as pd\r\n",
    "import numpy as np\r\n",
    "import matplotlib.pyplot as plt\r\n",
    "from tqdm import tqdm\r\n",
    "%matplotlib inline"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### **2.GPU Device Loaded**"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "source": [
    "device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### **3.Load and Prepare the Data**"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "source": [
    "# load the dataset\r\n",
    "train = pd.read_csv('https://pjreddie.com/media/files/mnist_train.csv')\r\n",
    "test = pd.read_csv('https://pjreddie.com/media/files/mnist_test.csv')"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "source": [
    "type(train)"
   ],
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "pandas.core.frame.DataFrame"
      ]
     },
     "metadata": {},
     "execution_count": 51
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "source": [
    "# get the image pixels and labels\r\n",
    "train_images = train.iloc[:,1:]\r\n",
    "train_labels = train.iloc[:,0]\r\n",
    "\r\n",
    "test_images = test.iloc[:,1:]\r\n",
    "test_labels = test.iloc[:,0]"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "source": [
    "print(train_images.shape, train_labels.shape)\r\n",
    "print(test_labels.shape, test_labels.shape)"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "(59999, 784) (59999,)\n",
      "(9999,) (9999,)\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### **4.Define the Transforms**"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "source": [
    "transform = transforms.Compose([\r\n",
    "    transforms.ToPILImage(),\r\n",
    "    transforms.ToTensor(),\r\n",
    "    transforms.Normalize((0.5,), (0.5,))\r\n",
    "])"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### **5.Prepare the Custom Dataset**"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "source": [
    "# Custom class\r\n",
    "class MNISTDataset(Dataset):\r\n",
    "    def __init__(self, images, labels = None, transforms = None):\r\n",
    "        self.X = images\r\n",
    "        self.y = labels\r\n",
    "        self.transforms = transforms\r\n",
    "        \r\n",
    "    def __len__(self):\r\n",
    "        return len(self.X)\r\n",
    "    \r\n",
    "    def __getitem__(self, index):\r\n",
    "        data = self.X.iloc[index, :]\r\n",
    "        data = np.asarray(data).astype(np.uint8).reshape(28,28,1)\r\n",
    "        \r\n",
    "        if self.transforms:\r\n",
    "            data = self.transforms(data)\r\n",
    "        \r\n",
    "        if self.y is not None:\r\n",
    "            return data, self.y[index]\r\n",
    "        else:\r\n",
    "            return data"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### **6. Dataset Reading and Load the dataset.**"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "source": [
    "# Create the dataset\r\n",
    "trainData = MNISTDataset(train_images, train_labels, transform)\r\n",
    "testData = MNISTDataset(test_images, test_labels, transform)\r\n",
    "\r\n",
    "# Data Loader\r\n",
    "trainLoader = DataLoader(trainData, batch_size = 128, shuffle=True)\r\n",
    "testLoader = DataLoader(testData, batch_size = 128, shuffle=True)"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "* In the `__init__()` function we initialize the images, labels, and transforms. Note that by default the labels and transforms parameters are None. We will pass them as arguments depending on our requirements for the project. \r\n",
    "\r\n",
    "* The `__len__()` function returns the length as usual.\r\n",
    "\r\n",
    "* Most of the work is being done in the `__getitem__()` function. So, we get the data on the index by index basis. For each index, we get the pixel data for the entire row. then we convert the data to Numpy array and reshape it into 28×28 gray-scale images. then Apply the transforms to the pixel data based on the transforms that we have defined earlier. Also, we return the pixel data along with the corresponding label.\r\n",
    "\r\n",
    "* The next few lines of code are fairly straightforward. We create two objects, trainData and testData for the MNISTDataset() class. We pass the image pixels, the image labels, and the transforms as arguments. we define the `trainLoader` and `testLoader`. You must find this line very similar to the directly getting the dataloader from the PyTorch MNIST dataset."
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### **7.Define the Neural Network, Train, and Test it**"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "source": [
    "# define the neural net class\r\n",
    "class Net(nn.Module):\r\n",
    "    def __init__(self):\r\n",
    "        super(Net, self).__init__()\r\n",
    "        self.conv1 = nn.Conv2d(in_channels=1, out_channels=20, kernel_size=5, stride=1)\r\n",
    "        self.conv2 = nn.Conv2d(in_channels=20, out_channels=50, kernel_size=5, stride=1)\r\n",
    "        self.fc1 = nn.Linear(in_features=800, out_features=500)\r\n",
    "        self.fc2 = nn.Linear(in_features=500, out_features=10)\r\n",
    "    def forward(self, x):\r\n",
    "        x = F.relu(self.conv1(x))\r\n",
    "        x = F.max_pool2d(x, 2, 2)\r\n",
    "        x = F.relu(self.conv2(x))\r\n",
    "        x = F.max_pool2d(x, 2, 2)\r\n",
    "        x = x.view(x.size(0), -1)\r\n",
    "        x = F.relu(self.fc1(x))\r\n",
    "        x = self.fc2(x)\r\n",
    "        return x\r\n",
    "\r\n",
    "net = Net().to(device)\r\n",
    "print(net)"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "Net(\n",
      "  (conv1): Conv2d(1, 20, kernel_size=(5, 5), stride=(1, 1))\n",
      "  (conv2): Conv2d(20, 50, kernel_size=(5, 5), stride=(1, 1))\n",
      "  (fc1): Linear(in_features=800, out_features=500, bias=True)\n",
      "  (fc2): Linear(in_features=500, out_features=10, bias=True)\n",
      ")\n"
     ]
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "source": [
    "summary(net, (1, 28, 28))"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "==========================================================================================\n",
      "Layer (type:depth-idx)                   Output Shape              Param #\n",
      "==========================================================================================\n",
      "├─Conv2d: 1-1                            [-1, 20, 24, 24]          520\n",
      "├─Conv2d: 1-2                            [-1, 50, 8, 8]            25,050\n",
      "├─Linear: 1-3                            [-1, 500]                 400,500\n",
      "├─Linear: 1-4                            [-1, 10]                  5,010\n",
      "==========================================================================================\n",
      "Total params: 431,080\n",
      "Trainable params: 431,080\n",
      "Non-trainable params: 0\n",
      "Total mult-adds (M): 2.29\n",
      "==========================================================================================\n",
      "Input size (MB): 0.00\n",
      "Forward/backward pass size (MB): 0.12\n",
      "Params size (MB): 1.64\n",
      "Estimated Total Size (MB): 1.76\n",
      "==========================================================================================\n"
     ]
    },
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "==========================================================================================\n",
       "Layer (type:depth-idx)                   Output Shape              Param #\n",
       "==========================================================================================\n",
       "├─Conv2d: 1-1                            [-1, 20, 24, 24]          520\n",
       "├─Conv2d: 1-2                            [-1, 50, 8, 8]            25,050\n",
       "├─Linear: 1-3                            [-1, 500]                 400,500\n",
       "├─Linear: 1-4                            [-1, 10]                  5,010\n",
       "==========================================================================================\n",
       "Total params: 431,080\n",
       "Trainable params: 431,080\n",
       "Non-trainable params: 0\n",
       "Total mult-adds (M): 2.29\n",
       "==========================================================================================\n",
       "Input size (MB): 0.00\n",
       "Forward/backward pass size (MB): 0.12\n",
       "Params size (MB): 1.64\n",
       "Estimated Total Size (MB): 1.76\n",
       "=========================================================================================="
      ]
     },
     "metadata": {},
     "execution_count": 58
    }
   ],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### **8. Calculate the Loss Function and Optimization**"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "source": [
    "# loss\r\n",
    "criterion = nn.CrossEntropyLoss()\r\n",
    "# optimizer\r\n",
    "optimizer = optim.SGD(net.parameters(), lr=0.001, momentum=0.9)"
   ],
   "outputs": [],
   "metadata": {}
  },
  {
   "cell_type": "markdown",
   "source": [
    "### **9.training and test function and execute the functions**"
   ],
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "source": [
    "def train(net, trainloader):\r\n",
    "    for epoch in range(10): # no. of epochs\r\n",
    "        running_loss = 0\r\n",
    "        with tqdm(trainLoader, unit=\"batch\") as tepoch:\r\n",
    "            for data in tepoch:\r\n",
    "                tepoch.set_description(f\"Epoch {epoch+1}\")\r\n",
    "                # data pixels and labels to GPU if available\r\n",
    "                inputs, labels = data[0].to(device, non_blocking=True), data[1].to(device, non_blocking=True)\r\n",
    "                # set the parameter gradients to zero\r\n",
    "                optimizer.zero_grad()\r\n",
    "                outputs = net(inputs)\r\n",
    "                loss = criterion(outputs, labels)\r\n",
    "                # propagate the loss backward\r\n",
    "                loss.backward()\r\n",
    "                # update the gradients\r\n",
    "                optimizer.step()\r\n",
    "                tepoch.set_postfix(loss=loss)\r\n",
    "                running_loss += loss.item()\r\n",
    "                \r\n",
    "            print('[Epoch %d] loss: %.3f' %(epoch + 1, running_loss/len(trainloader)))\r\n",
    " \r\n",
    "    print('Done Training')\r\n",
    "    \r\n",
    "    \r\n",
    "def test(net, testloader):\r\n",
    "    correct = 0\r\n",
    "    total = 0\r\n",
    "    with torch.no_grad():\r\n",
    "        for data in testLoader:\r\n",
    "            inputs, labels = data[0].to(device, non_blocking=True), data[1].to(device, non_blocking=True)\r\n",
    "            outputs = net(inputs)\r\n",
    "            _, predicted = torch.max(outputs.data, 1)\r\n",
    "            total += labels.size(0)\r\n",
    "            correct += (predicted == labels).sum().item()\r\n",
    "    print('Accuracy of the network on test images: %0.3f %%' % (100 * correct / total))\r\n",
    "    \r\n",
    "train(net, trainLoader)\r\n",
    "test(net, testLoader)"
   ],
   "outputs": [
    {
     "output_type": "stream",
     "name": "stderr",
     "text": [
      "Epoch 1: 100%|██████████| 469/469 [00:35<00:00, 13.31batch/s, loss=tensor(0.3137, device='cuda:0', grad_fn=<NllLossBackward>)]\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "[Epoch 1] loss: 0.917\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stderr",
     "text": [
      "Epoch 2: 100%|██████████| 469/469 [00:29<00:00, 15.92batch/s, loss=tensor(0.1925, device='cuda:0', grad_fn=<NllLossBackward>)]\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "[Epoch 2] loss: 0.249\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stderr",
     "text": [
      "Epoch 3: 100%|██████████| 469/469 [00:28<00:00, 16.47batch/s, loss=tensor(0.1427, device='cuda:0', grad_fn=<NllLossBackward>)]\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "[Epoch 3] loss: 0.168\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stderr",
     "text": [
      "Epoch 4: 100%|██████████| 469/469 [00:29<00:00, 16.08batch/s, loss=tensor(0.1017, device='cuda:0', grad_fn=<NllLossBackward>)]\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "[Epoch 4] loss: 0.129\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stderr",
     "text": [
      "Epoch 5: 100%|██████████| 469/469 [00:28<00:00, 16.68batch/s, loss=tensor(0.1203, device='cuda:0', grad_fn=<NllLossBackward>)]\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "[Epoch 5] loss: 0.107\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stderr",
     "text": [
      "Epoch 6: 100%|██████████| 469/469 [00:30<00:00, 15.63batch/s, loss=tensor(0.0378, device='cuda:0', grad_fn=<NllLossBackward>)]\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "[Epoch 6] loss: 0.093\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stderr",
     "text": [
      "Epoch 7: 100%|██████████| 469/469 [00:29<00:00, 16.15batch/s, loss=tensor(0.0912, device='cuda:0', grad_fn=<NllLossBackward>)]\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "[Epoch 7] loss: 0.083\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stderr",
     "text": [
      "Epoch 8: 100%|██████████| 469/469 [00:26<00:00, 17.89batch/s, loss=tensor(0.0299, device='cuda:0', grad_fn=<NllLossBackward>)]\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "[Epoch 8] loss: 0.076\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stderr",
     "text": [
      "Epoch 9: 100%|██████████| 469/469 [00:23<00:00, 19.57batch/s, loss=tensor(0.0840, device='cuda:0', grad_fn=<NllLossBackward>)]\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "[Epoch 9] loss: 0.070\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stderr",
     "text": [
      "Epoch 10: 100%|██████████| 469/469 [00:26<00:00, 17.81batch/s, loss=tensor(0.0777, device='cuda:0', grad_fn=<NllLossBackward>)]\n"
     ]
    },
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "[Epoch 10] loss: 0.065\n",
      "Done Training\n",
      "Accuracy of the network on test images: 98.220 %\n"
     ]
    }
   ],
   "metadata": {}
  }
 ],
 "metadata": {
  "orig_nbformat": 4,
  "language_info": {
   "name": "python",
   "version": "3.8.0",
   "mimetype": "text/x-python",
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "pygments_lexer": "ipython3",
   "nbconvert_exporter": "python",
   "file_extension": ".py"
  },
  "kernelspec": {
   "name": "python3",
   "display_name": "Python 3.8.0 64-bit ('pytorch19': conda)"
  },
  "interpreter": {
   "hash": "884a8f81666a19c0851426c83cd6eaa7b212468ad852fb3caa21591c98d4369f"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}