Natural_Language_Processing.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "toc_visible": true,
      "authorship_tag": "ABX9TyNgmE+1taRAlX4BG+UMfhzb",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/dr-mushtaq/Natural-language-processing/blob/main/Natural_Language_Processing.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "FF6zW3gnDvfA"
      },
      "outputs": [],
      "source": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **Table of Content**\n",
        "\n",
        "\n",
        "1.   Text Classification\n",
        "2.   Machine Translation\n",
        "3.   Name Entity Recognition\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "dnKZQpZJghaC"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **Import Libraries**"
      ],
      "metadata": {
        "id": "W_om6ncoEdax"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "This Portion is intended to test all the dependencies that are needed for the rest of the notebooks to run properly. If everything is tested correctly you should see a \"You are all set\" message when running the code below"
      ],
      "metadata": {
        "id": "IrdA4T7iEi1i"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import os\n",
        "from nltk.tokenize import word_tokenize\n",
        "from nltk.corpus import stopwords\n",
        "from nltk import ngrams\n",
        "import string\n",
        "from nltk.stem import PorterStemmer\n",
        "import matplotlib.pyplot as plt\n",
        "from wordcloud import WordCloud\n",
        "from collections import Counter\n",
        "from sklearn.feature_extraction.text import TfidfVectorizer\n",
        "from sklearn.svm import LinearSVC\n",
        "from sklearn.feature_extraction.text import TfidfVectorizer\n",
        "from sklearn.svm import LinearSVC\n",
        "from pprint import pprint\n",
        "from sklearn.preprocessing import MultiLabelBinarizer\n",
        "from sklearn.multiclass import OneVsRestClassifier\n",
        "from sklearn.metrics import f1_score, precision_score, recall_score\n",
        "from nltk.corpus import reuters\n",
        "from operator import itemgetter"
      ],
      "metadata": {
        "id": "7WYqdtPVEueW"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "import nltk\n",
        "nltk.download('stopwords')\n",
        "nltk.download('punkt')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "lHrl36fSE72Z",
        "outputId": "eca46d46-f9b2-46b4-db4a-a0fc531c9203"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[nltk_data] Downloading package stopwords to /root/nltk_data...\n",
            "[nltk_data]   Unzipping corpora/stopwords.zip.\n",
            "[nltk_data] Downloading package punkt to /root/nltk_data...\n",
            "[nltk_data]   Unzipping tokenizers/punkt.zip.\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "metadata": {},
          "execution_count": 2
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Test that verifies that the imports don't fail and that we have the correct files from nltk downloaded\n",
        "\n",
        "stopwords.words('english')\n",
        "word_tokenize('This is just a test')\n",
        "\n",
        "print('You are all set')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Bp7awaDVE25Y",
        "outputId": "94889271-2445-47b4-fd4e-ef2c0b4ddd3a"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "You are all set\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **📚1- Text Classification**"
      ],
      "metadata": {
        "id": "tpEx53qfD2Mq"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## **Table of Coentent**\n",
        "\n",
        "\n",
        "1.   Binary text Classification\n",
        "2.   Multi-Class Classification\n",
        "\n"
      ],
      "metadata": {
        "id": "yhbl2qw4hO1F"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## **📚1.1-Binary text classification**"
      ],
      "metadata": {
        "id": "S-BPs6llHZgJ"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "We will address the binary problem of detecting Sport related documents vs any other type of documents. In order to do this we will create an artificial (and very small collection).\n",
        "\n",
        "Define a set of labelled documents that will be our training dataset. These are the documents the classifier will learn from in order to categorise future unseen documents\n",
        "\n",
        "Define a set of labelled documents that will be our testing dataset. These will be the \"unseen\" documents that the classifier will predict (without having being trained with them)\n",
        "\n",
        "Represent our training and testing documents\n",
        "\n",
        "Train the classifier based on the training data\n",
        "\n",
        "Predict the labels for the testing documents"
      ],
      "metadata": {
        "id": "Pz1m1RJUHg3e"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "###**📚1.1.1-Support Vector Machine (SVM)**"
      ],
      "metadata": {
        "id": "JJXnHE1aHxP-"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### **Import library**"
      ],
      "metadata": {
        "id": "BgHdV8sLy8rD"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.feature_extraction.text import TfidfVectorizer\n",
        "from sklearn.svm import LinearSVC"
      ],
      "metadata": {
        "id": "z4f61c9bHapn"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Train and test data. Both the full documents and their labels (\"Sports\" vs \"Non Sports\")\n",
        "train_data = ['Football: a great sport', 'The referee has been very bad this season', 'Our team scored 5 goals', 'I love tennis',\n",
        "              'Politics is in decline in the UK', 'Brexit means Brexit', 'The parlament wants to create new legislation',\n",
        "              'I so want to travel the world']\n",
        "train_labels = [\"Sports\",\"Sports\",\"Sports\",\"Sports\", \"Non Sports\", \"Non Sports\", \"Non Sports\", \"Non Sports\"]\n",
        "\n",
        "test_data = ['Swimming is a great sport',\n",
        "             'A lot of policy changes will happen after Brexit',\n",
        "             'The table tennis team will travel to the UK soon for the European Championship']\n",
        "test_labels = [\"Sports\",\"Non Sports\",\"Sports\"]"
      ],
      "metadata": {
        "id": "XN7EkSTNH_pl"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### **Data Preprocessing and model training**"
      ],
      "metadata": {
        "id": "DYK9S5VPzKlK"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Representation of the data using TF-IDF\n",
        "vectorizer = TfidfVectorizer()\n",
        "vectorised_train_data = vectorizer.fit_transform(train_data)\n",
        "vectorised_test_data = vectorizer.transform(test_data)\n",
        "\n",
        "# Train the classifier given the training data\n",
        "classifier = LinearSVC()\n",
        "classifier.fit(vectorised_train_data, train_labels)\n",
        "\n",
        "# Predict the labels for the test documents (not used for training)\n",
        "print(classifier.predict(vectorised_test_data))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "FEsBpwm-IHQk",
        "outputId": "559642b2-6838-4f92-9e82-dddf2854eaa4"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "['Sports' 'Non Sports' 'Non Sports']\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "However, the third case is wrongly classified. Why do you think that might be?\n",
        "\n",
        "Matching problems (e.g., \"car\" is different than \"Cars\")\n",
        "\n",
        "Cases never seen before (e.g., the classifier has never seen the word \"table\")\n",
        "\n",
        "\"Spurious\" correlations and bias (\"car\" appears only in the positive category)"
      ],
      "metadata": {
        "id": "7rco0gUfJTsm"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Lets look into how we are representing our documents\n",
        "\n"
      ],
      "metadata": {
        "id": "XpRX2oHkJhRM"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### **Model Performance improment**"
      ],
      "metadata": {
        "id": "59djFza5dsAO"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from pprint import pprint\n",
        "\n",
        "# Function to show the feature weights of a document (to be explained later)\n",
        "def feature_values(doc, representer):\n",
        "    doc_representation = representer.transform([doc])\n",
        "    features = representer.get_feature_names()\n",
        "    return [(features[index], doc_representation[0, index]) for index in doc_representation.nonzero()[1]]\n",
        "\n",
        "pprint([feature_values(doc, vectorizer) for doc in test_data])"
      ],
      "metadata": {
        "id": "Y8EuXpoVJist"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Lets try again, with stop-word removal this time**"
      ],
      "metadata": {
        "id": "ITiaQF1XJx0T"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from nltk.corpus import stopwords\n",
        "\n",
        "# Load the list of (english) stop-words from nltk\n",
        "stop_words = stopwords.words(\"english\")\n",
        "\n",
        "# Represent, train, predict\n",
        "vectorizer = TfidfVectorizer(stop_words=stop_words)\n",
        "vectorised_train_data = vectorizer.fit_transform(train_data)\n",
        "vectorised_test_data = vectorizer.transform(test_data)\n",
        "classifier = LinearSVC()\n",
        "classifier.fit(vectorised_train_data, train_labels)\n",
        "binary_predictions=classifier.predict(vectorised_test_data)\n",
        "\n",
        "print(classifier.predict(vectorised_test_data))\n",
        "# Expected: [Sports, Non Sports, Sports]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "pBqlkEhmJ1Ri",
        "outputId": "74649bdb-79b1-437b-f9c9-dfff75986c98"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "['Sports' 'Non Sports' 'Sports']\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### **Model Evaluation**"
      ],
      "metadata": {
        "id": "rd1f5q9cnYNe"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.metrics import f1_score, precision_score, recall_score\n",
        "\n",
        "# Binary problem\n",
        "binary_labels = [1, 0, 1]\n",
        "binary_predictions = [1, 0, 0]\n",
        "\n",
        "# Quality values (with respect to class 1 by default)\n",
        "# Show our quality\n",
        "precision = precision_score(binary_labels, binary_predictions)\n",
        "recall = recall_score(binary_labels, binary_predictions)\n",
        "f1 = f1_score(binary_labels, binary_predictions)\n",
        "print(\"Micro-average quality numbers\")\n",
        "print(\"Precision: {:.4f}, Recall: {:.4f}, F1-measure: {:.4f}\".format(precision,\n",
        "                                                                     recall,\n",
        "                                                                     f1))\n",
        "\n",
        "binary_labels = [\"A\", \"B\", \"A\"]\n",
        "binary_predictions = [\"A\", \"B\", \"B\"]\n",
        "precision = precision_score(binary_labels, binary_predictions, pos_label=\"A\")\n",
        "recall = recall_score(binary_labels, binary_predictions, pos_label=\"A\")\n",
        "f1 = f1_score(binary_labels, binary_predictions, pos_label=\"A\")\n",
        "print(\"Precision: {:.4f}, Recall: {:.4f}, F1-measure: {:.4f}\".format(precision,\n",
        "                                                                     recall,\n",
        "                                                                     f1))"
      ],
      "metadata": {
        "id": "6UBg9wVrnLrf",
        "outputId": "34d09d65-3107-4445-cbee-4b20bc78fb1c",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Micro-average quality numbers\n",
            "Precision: 1.0000, Recall: 0.5000, F1-measure: 0.6667\n",
            "Precision: 1.0000, Recall: 0.5000, F1-measure: 0.6667\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## **📚1.2-Multi-Class classification problem**"
      ],
      "metadata": {
        "id": "NR6yeieEoKha"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### **📚1.2.1-Detecting the Language of senence**\n",
        "\n"
      ],
      "metadata": {
        "id": "nJI4XpNjtS0w"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### **1.2.1-Support Vector Machine (SVM)**"
      ],
      "metadata": {
        "id": "ll-uyOBlrB8j"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "We will address the multi-class problem of detecting the language of a sentence based on 3 mutually exclusive languages (e.g., Spanish, English and French). For the sake of this example, we assume those are the only 3 languages that the documents can have. As before, we will create an artificial (and very small collection) with similar steps"
      ],
      "metadata": {
        "id": "b9L3IGcHobW_"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.feature_extraction.text import TfidfVectorizer\n",
        "from sklearn.svm import LinearSVC"
      ],
      "metadata": {
        "id": "l1VVzPISowe-"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Artificial (and small) dataset. Spanish,English,French texts\n",
        "train_data = ['PyCon es una gran conferencia', 'Aprendizaje automatico esta listo para dominar el mundo dentro de poco',\n",
        "             'This is a great conference with a lot of amazing talks', 'AI will dominate the world in the near future',\n",
        "             'Dix chiffres por resumer le feuilleton de la loi travail']\n",
        "train_labels = [\"SP\", \"SP\", \"EN\", \"EN\", \"FR\"]\n",
        "\n",
        "test_data = ['Estoy preparandome para dominar las olimpiadas', 'Me gustaria mucho aprender el lenguage de programacion Scala',\n",
        "             'Machine Learning is amazing','Hola a todos']\n",
        "test_labels = [\"SP\", \"SP\", \"EN\", \"SP\"]"
      ],
      "metadata": {
        "id": "5rKrmOGCofBQ"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Represent\n",
        "vectorizer = TfidfVectorizer() # Note, we are not doing stop-word removal. Stop words could be beneficial in this problems\n",
        "vectorised_train_data = vectorizer.fit_transform(train_data)\n",
        "vectorised_test_data = vectorizer.transform(test_data)\n"
      ],
      "metadata": {
        "id": "snORR13fooXM"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Train\n",
        "classifier = LinearSVC()\n",
        "classifier.fit(vectorised_train_data, train_labels)"
      ],
      "metadata": {
        "id": "Idwy7PfMo3tn",
        "outputId": "18fef0bc-3eaa-4908-fd2b-db93802fbf67",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 75
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "LinearSVC()"
            ],
            "text/html": [
              "<style>#sk-container-id-1 {color: black;background-color: white;}#sk-container-id-1 pre{padding: 0;}#sk-container-id-1 div.sk-toggleable {background-color: white;}#sk-container-id-1 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-1 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-1 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-1 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-1 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-1 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-1 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-1 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-1 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-1 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-1 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-1 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-1 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-1 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-1 div.sk-item {position: relative;z-index: 1;}#sk-container-id-1 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-1 div.sk-item::before, #sk-container-id-1 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-1 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-1 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-1 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-1 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-1 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-1 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-1 div.sk-label-container {text-align: center;}#sk-container-id-1 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-1 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-1\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>LinearSVC()</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-1\" type=\"checkbox\" checked><label for=\"sk-estimator-id-1\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">LinearSVC</label><div class=\"sk-toggleable__content\"><pre>LinearSVC()</pre></div></div></div></div></div>"
            ]
          },
          "metadata": {},
          "execution_count": 16
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from pprint import pprint"
      ],
      "metadata": {
        "id": "BAKDwkHJpE22"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Predict\n",
        "predictions = classifier.predict(vectorised_test_data)\n",
        "pprint(predictions)"
      ],
      "metadata": {
        "id": "XyuaXY_xo8YG",
        "outputId": "0d370cae-3224-4833-cc82-30ac6eb86be3",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "array(['SP', 'SP', 'EN', 'EN'], dtype='<U2')\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "The last case is wrong. Can you guess why?\n",
        "\n",
        "Can we learn from never seen cases?\n"
      ],
      "metadata": {
        "id": "u3hXE4erpNoa"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "We will address the multi-label problem of labeling documents as being relevant to Sports or Politics. As before, we will create an artificial (and very small collection) with initial similar steps."
      ],
      "metadata": {
        "id": "PQ4qU73NpacM"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "There are two modifications for our example to run in a multi-label way:\n",
        "\n",
        "- Change the representation of the data viewing every document as a list of bits, representing being or not to each category. (MultiLabelBinarizer)\n",
        "\n",
        "- Run the classifier N times, once for each category where the negative cases will be the documents in all the other categories. (OneVsRestClassifier)"
      ],
      "metadata": {
        "id": "mF3nPNfapgOF"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.preprocessing import MultiLabelBinarizer\n",
        "from sklearn.multiclass import OneVsRestClassifier"
      ],
      "metadata": {
        "id": "D7ZiOTsYpfdc"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Artificial (and small) dataset. Sports and Politics\n",
        "train_data = ['Football: a great sport', 'The referee has been very bad this season', 'Our team scored 5 goals', 'I love tennis',\n",
        "              'Politics is in decline in the UK', 'Brexit means Brexit', 'The parlament wants to create new legislation',\n",
        "              'I so want to travel the world',\n",
        "              'The goverment will increase the budget for sports in the UK after the victories in the Olimpic Games',\n",
        "              \"O'Reilly has a great conference this year\"]\n",
        "train_labels = [[\"Sports\"], [\"Sports\"], [\"Sports\"], [\"Sports\"],[\"Politics\"],[\"Politics\"],[\"Politics\"],[],[\"Politics\", \"Sports\"],[]]\n",
        "\n",
        "test_data = ['Swimming is a great sport',\n",
        "             'A lot of policy changes will happen after Brexit',\n",
        "             'The table tennis team will travel to the UK soon for the European Championship',\n",
        "             'The goverment will increase the budget for sports in the UK after the victories in the Olimpic Games',\n",
        "             'PyCon is my favourite conference']\n",
        "test_labels = [[\"Sports\"], [\"Politics\"], [\"Sports\"], [\"Politics\",\"Sports\"],[]]"
      ],
      "metadata": {
        "id": "NS0lNTrAp4mL"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Change the representation of our data as a list of bit lists\n",
        "mlb = MultiLabelBinarizer()\n",
        "binary_train_labels = mlb.fit_transform(train_labels)\n",
        "binary_test_labels = mlb.transform(test_labels)\n",
        "\n",
        "print(binary_train_labels)"
      ],
      "metadata": {
        "id": "jaXXeZcrp8i0",
        "outputId": "dd7f77f7-0fa4-4e66-9602-81e212e00f5a",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[[0 0 0 1 0 1]\n",
            " [0 0 0 1 0 1]\n",
            " [1 0 1 0 0 0]\n",
            " [1 0 1 0 0 0]\n",
            " [0 1 0 0 1 0]]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import nltk\n",
        "nltk.download('stopwords')\n",
        "nltk.download('punkt')"
      ],
      "metadata": {
        "id": "wwamWJI3qXC7",
        "outputId": "b1040477-fd5a-4cdb-b989-4e55431b6f2d",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[nltk_data] Downloading package stopwords to /root/nltk_data...\n",
            "[nltk_data]   Package stopwords is already up-to-date!\n",
            "[nltk_data] Downloading package punkt to /root/nltk_data...\n",
            "[nltk_data]   Package punkt is already up-to-date!\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "metadata": {},
          "execution_count": 21
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from nltk.corpus import stopwords\n",
        "\n",
        "# Load the list of (english) stop-words from nltk\n",
        "stop_words = stopwords.words(\"english\")"
      ],
      "metadata": {
        "id": "PFJlx-Fzqm8S"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Represent\n",
        "vectorizer = TfidfVectorizer(stop_words=stop_words)\n",
        "vectorised_train_data = vectorizer.fit_transform(train_data)\n",
        "vectorised_test_data = vectorizer.transform(test_data)\n"
      ],
      "metadata": {
        "id": "j9uL7LM_qHpL"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# One classifer built per category using a one vs the rest approach\n",
        "classifier = OneVsRestClassifier(LinearSVC())\n",
        "classifier.fit(vectorised_train_data, binary_train_labels)"
      ],
      "metadata": {
        "id": "2hNu-wxBqrpl",
        "outputId": "19dcaa5d-809d-4f1b-b35f-d7841edfd421",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 118
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "OneVsRestClassifier(estimator=LinearSVC())"
            ],
            "text/html": [
              "<style>#sk-container-id-2 {color: black;background-color: white;}#sk-container-id-2 pre{padding: 0;}#sk-container-id-2 div.sk-toggleable {background-color: white;}#sk-container-id-2 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-2 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-2 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-2 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-2 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-2 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-2 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-2 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-2 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-2 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-2 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-2 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-2 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-2 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-2 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-2 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-2 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-2 div.sk-item {position: relative;z-index: 1;}#sk-container-id-2 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-2 div.sk-item::before, #sk-container-id-2 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-2 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-2 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-2 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-2 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-2 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-2 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-2 div.sk-label-container {text-align: center;}#sk-container-id-2 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-2 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-2\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>OneVsRestClassifier(estimator=LinearSVC())</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-2\" type=\"checkbox\" ><label for=\"sk-estimator-id-2\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">OneVsRestClassifier</label><div class=\"sk-toggleable__content\"><pre>OneVsRestClassifier(estimator=LinearSVC())</pre></div></div></div><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-3\" type=\"checkbox\" ><label for=\"sk-estimator-id-3\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">estimator: LinearSVC</label><div class=\"sk-toggleable__content\"><pre>LinearSVC()</pre></div></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-4\" type=\"checkbox\" ><label for=\"sk-estimator-id-4\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">LinearSVC</label><div class=\"sk-toggleable__content\"><pre>LinearSVC()</pre></div></div></div></div></div></div></div></div></div></div>"
            ]
          },
          "metadata": {},
          "execution_count": 19
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#Predict\n",
        "predictions = classifier.predict(vectorised_test_data)"
      ],
      "metadata": {
        "id": "xv7xok4wqv0j"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "print(predictions)\n",
        "print()"
      ],
      "metadata": {
        "id": "5tlpTAzoqzuc",
        "outputId": "3f1b72db-ad85-4110-c460-a3c9dd79386b",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[[0 1]\n",
            " [1 0]\n",
            " [0 1]\n",
            " [1 1]\n",
            " [0 0]]\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "print(mlb.inverse_transform(predictions))\n"
      ],
      "metadata": {
        "id": "gnCG6w7Gq35y"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### **1.2.2-Naive Bayes classifie**"
      ],
      "metadata": {
        "id": "3wwLtRxYVwRE"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.datasets import fetch_20newsgroups\n",
        "from sklearn.feature_extraction.text import TfidfVectorizer\n",
        "from sklearn.naive_bayes import MultinomialNB\n",
        "from sklearn.metrics import accuracy_score"
      ],
      "metadata": {
        "id": "rBX8XhTXVmQC"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Load the 20 Newsgroups dataset\n",
        "newsgroups_train = fetch_20newsgroups(subset='train')\n",
        "newsgroups_test = fetch_20newsgroups(subset='test')"
      ],
      "metadata": {
        "id": "IVjyi0M9Vtve"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "newsgroups_test"
      ],
      "metadata": {
        "id": "m3bUCLrWWVhf"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Transform the texts into TF-IDF vectors\n",
        "vectorizer = TfidfVectorizer()\n",
        "X_train = vectorizer.fit_transform(newsgroups_train.data)\n",
        "X_test = vectorizer.transform(newsgroups_test.data)\n",
        "y_train = newsgroups_train.target\n",
        "y_test = newsgroups_test.target"
      ],
      "metadata": {
        "id": "Fg0ecCIoWD-I"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Train a Multinomial Naive Bayes classifier\n",
        "clf = MultinomialNB()\n",
        "clf.fit(X_train, y_train)"
      ],
      "metadata": {
        "id": "y2xZG1OvWRoe"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Predict the newsgroup of the test texts\n",
        "y_pred = clf.predict(X_test)"
      ],
      "metadata": {
        "id": "YRr1zyKZWmz3"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Evaluate the classifier's accuracy\n",
        "accuracy = accuracy_score(y_test, y_pred)\n",
        "print(\"Accuracy: {:.2f}%\".format(accuracy * 100))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "87P7RL51Wq7m",
        "outputId": "8bd2e174-d36d-43ee-9297-29359133bbbc"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Accuracy: 77.39%\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "This code will load the 20 Newsgroups dataset and split it into training and test sets. Then it will transform the texts into numerical representation using TfidfVectorizer and train a Multinomial Naive Bayes classifier using the training set. Finally, it will use the trained classifier to predict the newsgroup of the test texts and evaluate the classifier’s accuracy."
      ],
      "metadata": {
        "id": "-pWH5AQDWytG"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### **1.2.3-Logistic Regression**"
      ],
      "metadata": {
        "id": "6i967K17qorO"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.pipeline import Pipeline\n",
        "from sklearn.feature_extraction.text import CountVectorizer, TfidfTransformer"
      ],
      "metadata": {
        "id": "OEUd0euIrGpG"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.linear_model import LogisticRegression as LR"
      ],
      "metadata": {
        "id": "eChjVQG3q79_"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "lr_clf = Pipeline([('vect', CountVectorizer()), ('tfidf', TfidfTransformer()), ('clf',LR())])"
      ],
      "metadata": {
        "id": "autpwTSsq_3d"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "%%time\n",
        "lr_clf.fit(X=train_data, y=train_labels) # note that .fit function calls are inplace, and the Pipeline is not re-assigned"
      ],
      "metadata": {
        "id": "rFSi5g3RrN2z",
        "outputId": "a171990b-0fb1-48da-ac6d-e3ea8fe39a25",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 219
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "CPU times: user 24.5 ms, sys: 99 µs, total: 24.6 ms\n",
            "Wall time: 61 ms\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Pipeline(steps=[('vect', CountVectorizer()), ('tfidf', TfidfTransformer()),\n",
              "                ('clf', LogisticRegression())])"
            ],
            "text/html": [
              "<style>#sk-container-id-2 {color: black;background-color: white;}#sk-container-id-2 pre{padding: 0;}#sk-container-id-2 div.sk-toggleable {background-color: white;}#sk-container-id-2 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-2 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-2 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-2 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-2 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-2 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-2 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-2 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-2 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-2 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-2 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-2 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-2 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-2 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-2 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-2 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-2 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-2 div.sk-item {position: relative;z-index: 1;}#sk-container-id-2 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-2 div.sk-item::before, #sk-container-id-2 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-2 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-2 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-2 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-2 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-2 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-2 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-2 div.sk-label-container {text-align: center;}#sk-container-id-2 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-2 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-2\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>Pipeline(steps=[(&#x27;vect&#x27;, CountVectorizer()), (&#x27;tfidf&#x27;, TfidfTransformer()),\n",
              "                (&#x27;clf&#x27;, LogisticRegression())])</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-2\" type=\"checkbox\" ><label for=\"sk-estimator-id-2\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">Pipeline</label><div class=\"sk-toggleable__content\"><pre>Pipeline(steps=[(&#x27;vect&#x27;, CountVectorizer()), (&#x27;tfidf&#x27;, TfidfTransformer()),\n",
              "                (&#x27;clf&#x27;, LogisticRegression())])</pre></div></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-3\" type=\"checkbox\" ><label for=\"sk-estimator-id-3\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">CountVectorizer</label><div class=\"sk-toggleable__content\"><pre>CountVectorizer()</pre></div></div></div><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-4\" type=\"checkbox\" ><label for=\"sk-estimator-id-4\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">TfidfTransformer</label><div class=\"sk-toggleable__content\"><pre>TfidfTransformer()</pre></div></div></div><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-5\" type=\"checkbox\" ><label for=\"sk-estimator-id-5\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">LogisticRegression</label><div class=\"sk-toggleable__content\"><pre>LogisticRegression()</pre></div></div></div></div></div></div></div>"
            ]
          },
          "metadata": {},
          "execution_count": 26
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "lr_predicted = lr_clf.predict(test_data)"
      ],
      "metadata": {
        "id": "N-UQskEgs0qh"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "lr_acc = sum(lr_predicted == test_labels)/len(lr_predicted)"
      ],
      "metadata": {
        "id": "9ZbNY1aXs9EC"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "lr_acc"
      ],
      "metadata": {
        "id": "cYWGgh1XtH_M",
        "outputId": "4a2db7b6-f9b1-4294-bfc7-54c08c9b16cf",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.75"
            ]
          },
          "metadata": {},
          "execution_count": 31
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "def imdb_acc(pipeline_clf):\n",
        "    predictions = pipeline_clf.predict(test_data)\n",
        "    assert len(test_labels) == len(predictions)\n",
        "    return sum(predictions == test_labels)/len(test_labels), predictions"
      ],
      "metadata": {
        "id": "wZhWqi2_tRXH"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Remove Stop Words**"
      ],
      "metadata": {
        "id": "MXBkHX2et3b7"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "lr_clf = Pipeline([('vect', CountVectorizer(stop_words='english')), ('tfidf', TfidfTransformer()), ('clf',LR())])\n",
        "lr_clf.fit(X=train_data, y=train_labels)\n",
        "lr_acc, lr_predictions = imdb_acc(lr_clf)\n",
        "lr_acc"
      ],
      "metadata": {
        "id": "t7BX80DEtmN0",
        "outputId": "4bf10c39-18c1-4ab1-a73a-d1ef57933ef2",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.75"
            ]
          },
          "metadata": {},
          "execution_count": 33
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Increase the Ngram Range**"
      ],
      "metadata": {
        "id": "EVybsBftt6Ym"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "lr_clf = Pipeline([('vect', CountVectorizer(stop_words='english', ngram_range=(1,3))), ('tfidf', TfidfTransformer()), ('clf',LR())])\n",
        "lr_clf.fit(X=train_data, y=train_labels)\n",
        "lr_acc, lr_predictions = imdb_acc(lr_clf)\n",
        "lr_acc"
      ],
      "metadata": {
        "id": "tvLB1SXbuFuf",
        "outputId": "05dfbcef-2540-4ec3-a4e5-1c79d0f9c062",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "1.0"
            ]
          },
          "metadata": {},
          "execution_count": 34
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### **1.2.4- Tree Baseed Models**"
      ],
      "metadata": {
        "id": "CvWKO5rPxU-n"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Decision Trees**"
      ],
      "metadata": {
        "id": "ZrcHOLiRxtLG"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.tree import DecisionTreeClassifier as DTC\n",
        "dtc_clf = Pipeline([('vect', CountVectorizer()), ('tfidf', TfidfTransformer()), ('clf',DTC())])\n",
        "dtc_clf.fit(X=train_data, y=train_labels)\n",
        "dtc_acc, dtc_predictions = imdb_acc(dtc_clf)\n",
        "dtc_acc"
      ],
      "metadata": {
        "id": "0g_-GM0vx0Iz",
        "outputId": "b04e4100-a64f-4945-d64c-efe2c347bcf2",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.75"
            ]
          },
          "metadata": {},
          "execution_count": 35
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Random Forest Classifier**"
      ],
      "metadata": {
        "id": "XQMwx0N-yDaD"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.ensemble import RandomForestClassifier as RFC\n",
        "rfc_clf = Pipeline([('vect', CountVectorizer()), ('tfidf', TfidfTransformer()), ('clf',RFC())])\n",
        "rfc_clf.fit(X=train_data, y=train_labels)\n",
        "rfc_acc, rfc_predictions = imdb_acc(rfc_clf)\n",
        "rfc_acc"
      ],
      "metadata": {
        "id": "bbLRcLs9yJ4L",
        "outputId": "dfab79d9-4fd9-495a-d0f8-7c21fe59b019",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.75"
            ]
          },
          "metadata": {},
          "execution_count": 36
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Extra Trees Classifier**"
      ],
      "metadata": {
        "id": "AfCBsuf0zGaq"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.ensemble import ExtraTreesClassifier as XTC\n",
        "xtc_clf = Pipeline([('vect', CountVectorizer()), ('tfidf', TfidfTransformer()), ('clf',XTC())])\n",
        "xtc_clf.fit(X=train_data, y=train_labels)\n",
        "xtc_acc, xtc_predictions = imdb_acc(xtc_clf)\n",
        "xtc_acc"
      ],
      "metadata": {
        "id": "5kZX6BUmzLCG",
        "outputId": "ac741e2b-3631-4170-f880-5f4e50d9b4f7",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "1.0"
            ]
          },
          "metadata": {},
          "execution_count": 37
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### **1.2.5-Automatically Fine Tuning**"
      ],
      "metadata": {
        "id": "wlLCzQ3IzYsE"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**RandomizedSearch**"
      ],
      "metadata": {
        "id": "SqiOcqQ_zkh4"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.model_selection import RandomizedSearchCV\n"
      ],
      "metadata": {
        "id": "kdlF58s6zvL0"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.model_selection import RandomizedSearchCV\n",
        "param_grid = dict(clf__C=[50, 75, 85, 100],\n",
        "                  vect__stop_words=['english', None],\n",
        "                  vect__ngram_range = [(1, 1), (1, 3)],\n",
        "                  vect__lowercase = [True, False],\n",
        "                 )"
      ],
      "metadata": {
        "id": "QfDMrhYDz0LL"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "random_search = RandomizedSearchCV(lr_clf, param_distributions=param_grid, n_iter=5, scoring='accuracy', n_jobs=-1, cv=3)"
      ],
      "metadata": {
        "id": "WC-7EM0p0XlU"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "random_search.fit(train_data, train_labels)"
      ],
      "metadata": {
        "id": "BBtqZo5W0bq2"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### **📚1.2.2-Sentiment Analysis**\n"
      ],
      "metadata": {
        "id": "GLdTZyjrW7XV"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### **1.2.3-Text Pre-processing**"
      ],
      "metadata": {
        "id": "6jxwJl03tRv_"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Tokenization**\n"
      ],
      "metadata": {
        "id": "lkeYXjH9tsF7"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "text = 'Wisdoms daughter walks alone. The mark of Athena burns through rome'\n",
        "\n",
        "words = text.split()\n",
        "print(words)\n",
        "\n"
      ],
      "metadata": {
        "id": "uyX6l-7tt3HC",
        "outputId": "71331668-930c-493f-df62-6d51bd3dc33c",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "['Wisdoms', 'daughter', 'walks', 'alone.', 'The', 'mark', 'of', 'Athena', 'burns', 'through', 'rome']\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Normalisation**\n"
      ],
      "metadata": {
        "id": "EvzWQrAvuEEL"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "text = \"'To Sleep Or NOT to SLEep, THAT is THe Question'\"\n",
        "\n",
        "def lower_case(text):\n",
        "    text = text.lower()\n",
        "    return text\n",
        "\n",
        "lower_case = lower_case(text)#converts everthing to lowercase\n",
        "print(lower_case)"
      ],
      "metadata": {
        "id": "zvjpLGn-uLsc",
        "outputId": "d95a7238-9b96-4a81-b115-ce788265bc82",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "'to sleep or not to sleep, that is the question'\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from nltk.corpus import stopwords\n",
        "\n",
        "# Load the list of (english) stop-words from nltk\n",
        "import nltk\n",
        "nltk.download('stopwords')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "FbDW9rZ5XWwd",
        "outputId": "ffe8ba32-c5dd-40e7-cba4-64964e6a3cdf"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[nltk_data] Downloading package stopwords to /root/nltk_data...\n",
            "[nltk_data]   Unzipping corpora/stopwords.zip.\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "metadata": {},
          "execution_count": 12
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import nltk\n",
        "nltk.download('vader_lexicon')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "DoC6WDHDXk4-",
        "outputId": "02519655-2665-4708-bc31-b4c949ea54b4"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[nltk_data] Downloading package vader_lexicon to /root/nltk_data...\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "metadata": {},
          "execution_count": 14
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import nltk\n",
        "from nltk.sentiment import SentimentIntensityAnalyzer\n",
        "\n",
        "# Initialize sentiment analyzer\n",
        "sia = SentimentIntensityAnalyzer()\n",
        "\n",
        "# Define text to be analyzed\n",
        "text = \"I love this product, it's amazing\"\n",
        "\n",
        "# Get sentiment score\n",
        "sentiment_score = sia.polarity_scores(text)\n",
        "\n",
        "# Print sentiment score\n",
        "print(sentiment_score)\n",
        "\n",
        "# Classify sentiment as positive, negative, or neutral\n",
        "if sentiment_score['compound'] > 0.5:\n",
        "    print(\"Positive sentiment\")\n",
        "elif sentiment_score['compound'] < -0.5:\n",
        "    print(\"Negative sentiment\")\n",
        "else:\n",
        "    print(\"Neutral sentiment\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "3jxFLr14XKHM",
        "outputId": "127b5528-163c-4f60-d49e-7cf42344c83b"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "{'neg': 0.0, 'neu': 0.273, 'pos': 0.727, 'compound': 0.8402}\n",
            "Positive sentiment\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**TF-IDF and Term Frequency in Text Analysis**"
      ],
      "metadata": {
        "id": "yAQR69ztMYsU"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.feature_extraction.text import TfidfVectorizer\n",
        "\n",
        "# Sample documents\n",
        "documents = [\n",
        "    \"The cat chased the mouse.\",\n",
        "    \"The mouse ran away.\"\n",
        "]\n",
        "\n",
        "# Create the TF-IDF vectorizer\n",
        "vectorizer = TfidfVectorizer()\n",
        "\n",
        "# Generate the TF-IDF matrix\n",
        "tfidf_matrix = vectorizer.fit_transform(documents)\n",
        "\n",
        "# Get the feature names (terms)\n",
        "terms = vectorizer.get_feature_names_out()\n",
        "\n",
        "# Print the TF-IDF scores for each term in each document\n",
        "for i, doc in enumerate(documents):\n",
        "    print(f\"Document {i+1}:\")\n",
        "    for j, term in enumerate(terms):\n",
        "        tfidf_score = tfidf_matrix[i, j]\n",
        "        if tfidf_score > 0:\n",
        "            print(f\"    {term}: {tfidf_score:.3f}\")"
      ],
      "metadata": {
        "id": "zo8pD5fpMipq",
        "outputId": "325abaf1-a19c-4395-a9c6-2225fa7d2d8c",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Document 1:\n",
            "    cat: 0.470\n",
            "    chased: 0.470\n",
            "    mouse: 0.334\n",
            "    the: 0.669\n",
            "Document 2:\n",
            "    away: 0.576\n",
            "    mouse: 0.410\n",
            "    ran: 0.576\n",
            "    the: 0.410\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### **📚1.2.4-Movie Reviews**"
      ],
      "metadata": {
        "id": "qUaCLVgapOO4"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "##### **1.2.4.1- Bayes theorem**"
      ],
      "metadata": {
        "id": "Ud4KagRsfH5m"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import numpy as np # linear algebra\n",
        "import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n",
        "import re # for regex\n",
        "from nltk.corpus import stopwords\n",
        "from nltk.tokenize import word_tokenize\n",
        "from nltk.stem import SnowballStemmer\n",
        "from sklearn.feature_extraction.text import CountVectorizer\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.naive_bayes import GaussianNB,MultinomialNB,BernoulliNB\n",
        "from sklearn.metrics import accuracy_score\n",
        "import pickle"
      ],
      "metadata": {
        "id": "Lh6HhDg9pi_u"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Dataset**"
      ],
      "metadata": {
        "id": "w_pZvHOPpoav"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from google.colab import drive\n",
        "drive.mount('/content/drive')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "v0CW_-lhqXMm",
        "outputId": "f91ace83-4d1f-4272-9734-670840058c69"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Mounted at /content/drive\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "data = pd.read_csv('/content/drive/MyDrive/Courses /Data Science /NLP/Datasets/IMDB-Dataset.csv')\n",
        "print(data.shape)\n",
        "data.head()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 224
        },
        "id": "N56Slz7WpsHn",
        "outputId": "bcdb1492-9233-4489-f72d-bef886ea28b1"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(50000, 2)\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                              review sentiment\n",
              "0  One of the other reviewers has mentioned that ...  positive\n",
              "1  A wonderful little production. <br /><br />The...  positive\n",
              "2  I thought this was a wonderful way to spend ti...  positive\n",
              "3  Basically there's a family where a little boy ...  negative\n",
              "4  Petter Mattei's \"Love in the Time of Money\" is...  positive"
            ],
            "text/html": [
              "\n",
              "\n",
              "  <div id=\"df-4ed7ad56-b36f-42f0-a135-21ba55ecd437\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>review</th>\n",
              "      <th>sentiment</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>One of the other reviewers has mentioned that ...</td>\n",
              "      <td>positive</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>A wonderful little production. &lt;br /&gt;&lt;br /&gt;The...</td>\n",
              "      <td>positive</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>I thought this was a wonderful way to spend ti...</td>\n",
              "      <td>positive</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>Basically there's a family where a little boy ...</td>\n",
              "      <td>negative</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>Petter Mattei's \"Love in the Time of Money\" is...</td>\n",
              "      <td>positive</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-4ed7ad56-b36f-42f0-a135-21ba55ecd437')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "\n",
              "\n",
              "\n",
              "    <div id=\"df-87aa0106-bcc2-45cd-a0b6-c500fe607b74\">\n",
              "      <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-87aa0106-bcc2-45cd-a0b6-c500fe607b74')\"\n",
              "              title=\"Suggest charts.\"\n",
              "              style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "      </button>\n",
              "    </div>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "    background-color: #E8F0FE;\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: #1967D2;\n",
              "    height: 32px;\n",
              "    padding: 0 0 0 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: #E2EBFA;\n",
              "    box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: #174EA6;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "    background-color: #3B4455;\n",
              "    fill: #D2E3FC;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart:hover {\n",
              "    background-color: #434B5C;\n",
              "    box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "    filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "    fill: #FFFFFF;\n",
              "  }\n",
              "</style>\n",
              "\n",
              "    <script>\n",
              "      async function quickchart(key) {\n",
              "        const containerElement = document.querySelector('#' + key);\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      }\n",
              "    </script>\n",
              "\n",
              "      <script>\n",
              "\n",
              "function displayQuickchartButton(domScope) {\n",
              "  let quickchartButtonEl =\n",
              "    domScope.querySelector('#df-87aa0106-bcc2-45cd-a0b6-c500fe607b74 button.colab-df-quickchart');\n",
              "  quickchartButtonEl.style.display =\n",
              "    google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "}\n",
              "\n",
              "        displayQuickchartButton(document);\n",
              "      </script>\n",
              "      <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-4ed7ad56-b36f-42f0-a135-21ba55ecd437 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-4ed7ad56-b36f-42f0-a135-21ba55ecd437');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n"
            ]
          },
          "metadata": {},
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "data.info()"
      ],
      "metadata": {
        "id": "fEzQn44hqwfn"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "data.sentiment.value_counts()"
      ],
      "metadata": {
        "id": "WDVnQ1RRq1pm"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "data.sentiment.replace('positive',1,inplace=True)\n",
        "data.sentiment.replace('negative',0,inplace=True)\n",
        "data.head(10)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 363
        },
        "id": "Q4mOmDSRq9uI",
        "outputId": "7400a512-8abb-4f24-a6f8-2123280722bb"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                              review  sentiment\n",
              "0  One of the other reviewers has mentioned that ...          1\n",
              "1  A wonderful little production. <br /><br />The...          1\n",
              "2  I thought this was a wonderful way to spend ti...          1\n",
              "3  Basically there's a family where a little boy ...          0\n",
              "4  Petter Mattei's \"Love in the Time of Money\" is...          1\n",
              "5  Probably my all-time favorite movie, a story o...          1\n",
              "6  I sure would like to see a resurrection of a u...          1\n",
              "7  This show was an amazing, fresh & innovative i...          0\n",
              "8  Encouraged by the positive comments about this...          0\n",
              "9  If you like original gut wrenching laughter yo...          1"
            ],
            "text/html": [
              "\n",
              "\n",
              "  <div id=\"df-89f4fb24-dd8c-40d4-9194-6b9f70cfd3d7\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>review</th>\n",
              "      <th>sentiment</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>One of the other reviewers has mentioned that ...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>A wonderful little production. &lt;br /&gt;&lt;br /&gt;The...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>I thought this was a wonderful way to spend ti...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>Basically there's a family where a little boy ...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>Petter Mattei's \"Love in the Time of Money\" is...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>Probably my all-time favorite movie, a story o...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>I sure would like to see a resurrection of a u...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>This show was an amazing, fresh &amp; innovative i...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>Encouraged by the positive comments about this...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>9</th>\n",
              "      <td>If you like original gut wrenching laughter yo...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-89f4fb24-dd8c-40d4-9194-6b9f70cfd3d7')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "\n",
              "\n",
              "\n",
              "    <div id=\"df-43bcc9a5-63ed-48a8-9a57-7b9105743114\">\n",
              "      <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-43bcc9a5-63ed-48a8-9a57-7b9105743114')\"\n",
              "              title=\"Suggest charts.\"\n",
              "              style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "      </button>\n",
              "    </div>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "    background-color: #E8F0FE;\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: #1967D2;\n",
              "    height: 32px;\n",
              "    padding: 0 0 0 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: #E2EBFA;\n",
              "    box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: #174EA6;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "    background-color: #3B4455;\n",
              "    fill: #D2E3FC;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart:hover {\n",
              "    background-color: #434B5C;\n",
              "    box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "    filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "    fill: #FFFFFF;\n",
              "  }\n",
              "</style>\n",
              "\n",
              "    <script>\n",
              "      async function quickchart(key) {\n",
              "        const containerElement = document.querySelector('#' + key);\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      }\n",
              "    </script>\n",
              "\n",
              "      <script>\n",
              "\n",
              "function displayQuickchartButton(domScope) {\n",
              "  let quickchartButtonEl =\n",
              "    domScope.querySelector('#df-43bcc9a5-63ed-48a8-9a57-7b9105743114 button.colab-df-quickchart');\n",
              "  quickchartButtonEl.style.display =\n",
              "    google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "}\n",
              "\n",
              "        displayQuickchartButton(document);\n",
              "      </script>\n",
              "      <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-89f4fb24-dd8c-40d4-9194-6b9f70cfd3d7 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-89f4fb24-dd8c-40d4-9194-6b9f70cfd3d7');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n"
            ]
          },
          "metadata": {},
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "data.review[0]"
      ],
      "metadata": {
        "id": "m_P-PXQxrCdm"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Pre-processing Steps**\n",
        "\n",
        "- Remove HTML tags\n",
        "- Remove special characters\n",
        "- Convert everything to lowercase\n",
        "- Remove stopwords\n",
        "- Stemming"
      ],
      "metadata": {
        "id": "9ZPwO-17rIt_"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "1. **Remove HTML tags**"
      ],
      "metadata": {
        "id": "5uGYrI-zrjKp"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def clean(text):\n",
        "    cleaned = re.compile(r'<.*?>')\n",
        "    return re.sub(cleaned,'',text)\n"
      ],
      "metadata": {
        "id": "PUe8A9Q7sRNc"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "data.review = data.review.apply(clean)\n",
        "data.review[0]"
      ],
      "metadata": {
        "id": "At_9I5TfsTMk"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**2. Remove special characters**"
      ],
      "metadata": {
        "id": "mjF11oldscsi"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def is_special(text):\n",
        "    rem = ''\n",
        "    for i in text:\n",
        "        if i.isalnum():\n",
        "            rem = rem + i\n",
        "        else:\n",
        "            rem = rem + ' '\n",
        "    return rem"
      ],
      "metadata": {
        "id": "x1kiM8BYsfdd"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "data.review = data.review.apply(is_special)\n",
        "data.review[0]"
      ],
      "metadata": {
        "id": "dMTQ8JyDsky6"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**3. Convert everything to lowercase**\n"
      ],
      "metadata": {
        "id": "WHrBYsJ4stEa"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def to_lower(text):\n",
        "    return text.lower()"
      ],
      "metadata": {
        "id": "U7i45e-Esy2y"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "data.review = data.review.apply(to_lower)\n",
        "data.review[0]"
      ],
      "metadata": {
        "id": "hEBKMVpws2_9"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**4. Remove stopwords**"
      ],
      "metadata": {
        "id": "AcJJPlDys9Zt"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import nltk\n",
        "nltk.download('stopwords')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Rtdx4A_vtOmd",
        "outputId": "ad25bfb3-d53e-4ab2-cfab-866b803b286b"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[nltk_data] Downloading package stopwords to /root/nltk_data...\n",
            "[nltk_data]   Unzipping corpora/stopwords.zip.\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "metadata": {},
          "execution_count": 17
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import nltk\n",
        "nltk.download('punkt')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "OwnxM-hwtxOY",
        "outputId": "43833468-8a82-444e-e535-1a83c158b577"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[nltk_data] Downloading package punkt to /root/nltk_data...\n",
            "[nltk_data]   Unzipping tokenizers/punkt.zip.\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "metadata": {},
          "execution_count": 19
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "def rem_stopwords(text):\n",
        "    stop_words = set(stopwords.words('english'))\n",
        "    words = word_tokenize(text)\n",
        "    return [w for w in words if w not in stop_words]"
      ],
      "metadata": {
        "id": "vcJTABWetCgC"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "data.review = data.review.apply(rem_stopwords)\n",
        "data.review[0]"
      ],
      "metadata": {
        "id": "85pZtqWUtFmn"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**5-Stem the words**"
      ],
      "metadata": {
        "id": "99yvPpRRt5P9"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def stem_txt(text):\n",
        "    ss = SnowballStemmer('english')\n",
        "    return \" \".join([ss.stem(w) for w in text])"
      ],
      "metadata": {
        "id": "unysduG1uGS6"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "data.review = data.review.apply(stem_txt)\n",
        "data.review[0]"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 174
        },
        "id": "I7rB_xSBuL7n",
        "outputId": "cbb4d287-f2ff-4a9a-96d1-db8c73cc1ae2"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'one review mention watch 1 oz episod hook right exact happen first thing struck oz brutal unflinch scene violenc set right word go trust show faint heart timid show pull punch regard drug sex violenc hardcor classic use word call oz nicknam given oswald maximum secur state penitentari focus main emerald citi experiment section prison cell glass front face inward privaci high agenda em citi home mani aryan muslim gangsta latino christian italian irish scuffl death stare dodgi deal shadi agreement never far away would say main appeal show due fact goe show dare forget pretti pictur paint mainstream audienc forget charm forget romanc oz mess around first episod ever saw struck nasti surreal say readi watch develop tast oz got accustom high level graphic violenc violenc injustic crook guard sold nickel inmat kill order get away well manner middl class inmat turn prison bitch due lack street skill prison experi watch oz may becom comfort uncomfort view that get touch darker side'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 22
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "data.head()"
      ],
      "metadata": {
        "id": "gkN-3LwQuXT3"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**CREATING THE MODEL**\n"
      ],
      "metadata": {
        "id": "0N79ZXcEulOg"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**1. Creating Bag Of Words (BOW)**\n"
      ],
      "metadata": {
        "id": "dNW_OhZauoOg"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "X = np.array(data.iloc[:,0].values)\n",
        "y = np.array(data.sentiment.values)\n",
        "cv = CountVectorizer(max_features = 1000)\n",
        "X = cv.fit_transform(data.review).toarray()\n",
        "print(\"X.shape = \",X.shape)\n",
        "print(\"y.shape = \",y.shape)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Mrk1mtuWusXw",
        "outputId": "98a8b818-3fcf-4973-d49c-d18753d84bce"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "X.shape =  (50000, 1000)\n",
            "y.shape =  (50000,)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**2. Train test split**"
      ],
      "metadata": {
        "id": "MmwHIoSdu1gL"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "trainx,testx,trainy,testy = train_test_split(X,y,test_size=0.2,random_state=9)\n",
        "print(\"Train shapes : X = {}, y = {}\".format(trainx.shape,trainy.shape))\n",
        "print(\"Test shapes : X = {}, y = {}\".format(testx.shape,testy.shape))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "z3wj2yHZu7N5",
        "outputId": "f762d3dd-e4ee-4213-88b1-a68c4e2a7796"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Train shapes : X = (40000, 1000), y = (40000,)\n",
            "Test shapes : X = (10000, 1000), y = (10000,)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**3. Defining the models and Training them**\n"
      ],
      "metadata": {
        "id": "GetBW37KvCdS"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "gnb,mnb,bnb = GaussianNB(),MultinomialNB(alpha=1.0,fit_prior=True),BernoulliNB(alpha=1.0,fit_prior=True)\n",
        "gnb.fit(trainx,trainy)\n",
        "mnb.fit(trainx,trainy)\n",
        "bnb.fit(trainx,trainy)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 75
        },
        "id": "qmW9MDpAvKR4",
        "outputId": "d79756e0-a1c6-4bfb-9fb1-4a28e29a0ffc"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "BernoulliNB()"
            ],
            "text/html": [
              "<style>#sk-container-id-1 {color: black;background-color: white;}#sk-container-id-1 pre{padding: 0;}#sk-container-id-1 div.sk-toggleable {background-color: white;}#sk-container-id-1 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-1 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-1 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-1 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-1 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-1 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-1 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-1 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-1 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-1 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-1 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-1 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-1 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-1 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-1 div.sk-item {position: relative;z-index: 1;}#sk-container-id-1 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-1 div.sk-item::before, #sk-container-id-1 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-1 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-1 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-1 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-1 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-1 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-1 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-1 div.sk-label-container {text-align: center;}#sk-container-id-1 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-1 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-1\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>BernoulliNB()</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-1\" type=\"checkbox\" checked><label for=\"sk-estimator-id-1\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">BernoulliNB</label><div class=\"sk-toggleable__content\"><pre>BernoulliNB()</pre></div></div></div></div></div>"
            ]
          },
          "metadata": {},
          "execution_count": 26
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**4. Prediction and accuracy metrics to choose best model**\n"
      ],
      "metadata": {
        "id": "Z6qNW4yZvQg8"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "ypg = gnb.predict(testx)\n",
        "ypm = mnb.predict(testx)\n",
        "ypb = bnb.predict(testx)\n",
        "\n",
        "print(\"Gaussian = \",accuracy_score(testy,ypg))\n",
        "print(\"Multinomial = \",accuracy_score(testy,ypm))\n",
        "print(\"Bernoulli = \",accuracy_score(testy,ypb))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "mUkHiMMNvb7c",
        "outputId": "b3bf9c84-7dd8-4694-c276-980989425acc"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Gaussian =  0.7843\n",
            "Multinomial =  0.831\n",
            "Bernoulli =  0.8386\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "pickle.dump(bnb,open('model1.pkl','wb'))"
      ],
      "metadata": {
        "id": "_Txnp5I6vhkO"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "rev =  \"\"\"Terrible. Complete trash. Brainless tripe. Insulting to anyone who isn't an 8 year old fan boy. Im actually pretty disgusted that this movie is making the money it is - what does it say about the people who brainlessly hand over the hard earned cash to be 'entertained' in this fashion and then come here to leave a positive 8.8 review?? Oh yes, they are morons. Its the only sensible conclusion to draw. How anyone can rate this movie amongst the pantheon of great titles is beyond me.\n",
        "\n",
        "So trying to find something constructive to say about this title is hard...I enjoyed Iron Man? Tony Stark is an inspirational character in his own movies but here he is a pale shadow of that...About the only 'hook' this movie had into me was wondering when and if Iron Man would knock Captain America out...Oh how I wished he had :( What were these other characters anyways? Useless, bickering idiots who really couldn't organise happy times in a brewery. The film was a chaotic mish mash of action elements and failed 'set pieces'...\n",
        "\n",
        "I found the villain to be quite amusing.\n",
        "\n",
        "And now I give up. This movie is not robbing any more of my time but I felt I ought to contribute to restoring the obvious fake rating and reviews this movie has been getting on IMDb.\"\"\"\n"
      ],
      "metadata": {
        "id": "ltNo82VRvnFQ"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "f1 = clean(rev)\n",
        "f2 = is_special(f1)\n",
        "f3 = to_lower(f2)\n",
        "f4 = rem_stopwords(f3)\n",
        "f5 = stem_txt(f4)"
      ],
      "metadata": {
        "id": "zyMB-CAavtqJ"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "bow,words = [],word_tokenize(f5)\n",
        "for word in words:\n",
        "    bow.append(words.count(word))\n",
        "#np.array(bow).reshape(1,3000)\n",
        "#bow.shape\n",
        "word_dict = cv.vocabulary_\n",
        "pickle.dump(word_dict,open('bow.pkl','wb'))"
      ],
      "metadata": {
        "id": "tvX4Spzjv4V8"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "inp = []\n",
        "for i in word_dict:\n",
        "    inp.append(f5.count(i[0]))\n",
        "y_pred = bnb.predict(np.array(inp).reshape(1,1000))"
      ],
      "metadata": {
        "id": "L3dZgXrov9C_"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "print(y_pred)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "v5u-EKYwwDs1",
        "outputId": "0ed7dfcc-37ba-489f-96d8-9e50904876d2"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[0]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### **📚1.2.3-Spam Detection**"
      ],
      "metadata": {
        "id": "60Sj9rM92Dv2"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "##### **1.2.3.1-Naive_bayes**"
      ],
      "metadata": {
        "id": "Qlr-cbyf3VLE"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import pandas as pd\n",
        "import numpy as np\n",
        "from sklearn.feature_extraction.text import CountVectorizer\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.naive_bayes import MultinomialNB\n"
      ],
      "metadata": {
        "id": "LGzLEyv82LhG"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "data = pd.read_csv(\"https://raw.githubusercontent.com/amankharwal/SMS-Spam-Detection/master/spam.csv\", encoding= 'latin-1')\n",
        "data.head()"
      ],
      "metadata": {
        "id": "2QgZSxSH2Sfe",
        "outputId": "2c108b91-e3d8-4929-9562-e9e7cca648a0",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "  class                                            message Unnamed: 2  \\\n",
              "0   ham  Go until jurong point, crazy.. Available only ...        NaN   \n",
              "1   ham                      Ok lar... Joking wif u oni...        NaN   \n",
              "2  spam  Free entry in 2 a wkly comp to win FA Cup fina...        NaN   \n",
              "3   ham  U dun say so early hor... U c already then say...        NaN   \n",
              "4   ham  Nah I don't think he goes to usf, he lives aro...        NaN   \n",
              "\n",
              "  Unnamed: 3 Unnamed: 4  \n",
              "0        NaN        NaN  \n",
              "1        NaN        NaN  \n",
              "2        NaN        NaN  \n",
              "3        NaN        NaN  \n",
              "4        NaN        NaN  "
            ],
            "text/html": [
              "\n",
              "\n",
              "  <div id=\"df-ce59f9f0-1391-4b30-913d-5f913eb267c1\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>class</th>\n",
              "      <th>message</th>\n",
              "      <th>Unnamed: 2</th>\n",
              "      <th>Unnamed: 3</th>\n",
              "      <th>Unnamed: 4</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>ham</td>\n",
              "      <td>Go until jurong point, crazy.. Available only ...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>ham</td>\n",
              "      <td>Ok lar... Joking wif u oni...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>spam</td>\n",
              "      <td>Free entry in 2 a wkly comp to win FA Cup fina...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>ham</td>\n",
              "      <td>U dun say so early hor... U c already then say...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>ham</td>\n",
              "      <td>Nah I don't think he goes to usf, he lives aro...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-ce59f9f0-1391-4b30-913d-5f913eb267c1')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "\n",
              "\n",
              "\n",
              "    <div id=\"df-c4d9ffe2-ab67-4d4c-8724-1a2c93bb584f\">\n",
              "      <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-c4d9ffe2-ab67-4d4c-8724-1a2c93bb584f')\"\n",
              "              title=\"Suggest charts.\"\n",
              "              style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "      </button>\n",
              "    </div>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "    background-color: #E8F0FE;\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: #1967D2;\n",
              "    height: 32px;\n",
              "    padding: 0 0 0 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: #E2EBFA;\n",
              "    box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: #174EA6;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "    background-color: #3B4455;\n",
              "    fill: #D2E3FC;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart:hover {\n",
              "    background-color: #434B5C;\n",
              "    box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "    filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "    fill: #FFFFFF;\n",
              "  }\n",
              "</style>\n",
              "\n",
              "    <script>\n",
              "      async function quickchart(key) {\n",
              "        const containerElement = document.querySelector('#' + key);\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      }\n",
              "    </script>\n",
              "\n",
              "      <script>\n",
              "\n",
              "function displayQuickchartButton(domScope) {\n",
              "  let quickchartButtonEl =\n",
              "    domScope.querySelector('#df-c4d9ffe2-ab67-4d4c-8724-1a2c93bb584f button.colab-df-quickchart');\n",
              "  quickchartButtonEl.style.display =\n",
              "    google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "}\n",
              "\n",
              "        displayQuickchartButton(document);\n",
              "      </script>\n",
              "      <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-ce59f9f0-1391-4b30-913d-5f913eb267c1 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-ce59f9f0-1391-4b30-913d-5f913eb267c1');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n"
            ]
          },
          "metadata": {},
          "execution_count": 32
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "data = data[[\"class\", \"message\"]]\n"
      ],
      "metadata": {
        "id": "JZ5Z_0Hh2XMS"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "x = np.array(data[\"message\"])\n",
        "y = np.array(data[\"class\"])"
      ],
      "metadata": {
        "id": "abgpovd12cce"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "In order to use textual data for predictive modeling, the text must be parsed to remove certain words – this process is called tokenization. These words need to then be encoded as integers, or floating-point values, for use as inputs in machine learning algorithms. This process is called feature extraction (or vectorization).\n",
        "\n",
        "Scikit-learn’s CountVectorizer is used to convert a collection of text documents to a vector of term/token counts. It also enables the ​pre-processing of text data prior to generating the vector representation. This functionality makes it a highly flexible feature representation module for text."
      ],
      "metadata": {
        "id": "okNlOyKG2ws4"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "cv = CountVectorizer()\n",
        "X = cv.fit_transform(x) # Fit the Data (# Encode the Document)"
      ],
      "metadata": {
        "id": "mKuSVT7121qH"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "print(x)"
      ],
      "metadata": {
        "id": "7Dm0OZT725it"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=42)\n"
      ],
      "metadata": {
        "id": "x0tkNF372_Du"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "clf = MultinomialNB()\n",
        "clf.fit(X_train,y_train)"
      ],
      "metadata": {
        "id": "eVtcuPJZ3CnH",
        "outputId": "38141c13-7dac-47b8-fef9-8905a898c1cd",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 75
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "MultinomialNB()"
            ],
            "text/html": [
              "<style>#sk-container-id-1 {color: black;background-color: white;}#sk-container-id-1 pre{padding: 0;}#sk-container-id-1 div.sk-toggleable {background-color: white;}#sk-container-id-1 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-1 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-1 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-1 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-1 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-1 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-1 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-1 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-1 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-1 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-1 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-1 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-1 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-1 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-1 div.sk-item {position: relative;z-index: 1;}#sk-container-id-1 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-1 div.sk-item::before, #sk-container-id-1 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-1 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-1 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-1 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-1 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-1 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-1 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-1 div.sk-label-container {text-align: center;}#sk-container-id-1 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-1 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-1\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>MultinomialNB()</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-1\" type=\"checkbox\" checked><label for=\"sk-estimator-id-1\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">MultinomialNB</label><div class=\"sk-toggleable__content\"><pre>MultinomialNB()</pre></div></div></div></div></div>"
            ]
          },
          "metadata": {},
          "execution_count": 39
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "sample = input('Enter a message:')\n",
        "data = cv.transform([sample]).toarray()\n",
        "print(clf.predict(data))"
      ],
      "metadata": {
        "id": "tKnhiwN73Iwm",
        "outputId": "62fa4c5d-9bf5-4d97-b1f0-fc2d49c9fd1d",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Enter a message:i am your friend\n",
            "['ham']\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "##### **1.2.3-2 Support Vector Machine**"
      ],
      "metadata": {
        "id": "sxgmbr580L06"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Detect emails whether spam or ham using Support Vector Machine and**"
      ],
      "metadata": {
        "id": "L6C_6_-t0dsE"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "###### I**mport Relevant Libraries**\n",
        "\n"
      ],
      "metadata": {
        "id": "3PeKsFVm0rAK"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import pandas as pd\n",
        "import numpy as np\n",
        "from sklearn.model_selection import train_test_split , GridSearchCV , KFold\n",
        "from sklearn.svm import SVC\n",
        "from sklearn.metrics import accuracy_score , classification_report , confusion_matrix\n",
        "import seaborn as sns\n",
        "import matplotlib.pyplot as plt\n",
        "import re\n",
        "import nltk\n",
        "from nltk.stem import PorterStemmer\n",
        "from sklearn import metrics\n",
        "from sklearn.feature_extraction.text import CountVectorizer"
      ],
      "metadata": {
        "id": "jn_z4Z-j07EH"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "###### **Import dataset from GitHub**\n",
        "\n"
      ],
      "metadata": {
        "id": "lF9AApKq1FRR"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "df = pd.read_csv(\"https://raw.githubusercontent.com/Sanjay-dev-ds/spam_ham_email_detector/master/spam.csv\")\n",
        "df.head()"
      ],
      "metadata": {
        "id": "z0Du3gps1Jrt",
        "outputId": "757aebf1-04f3-4702-ffaa-d383f1a066d1",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "  Label                                          EmailText\n",
              "0   ham  Go until jurong point, crazy.. Available only ...\n",
              "1   ham                      Ok lar... Joking wif u oni...\n",
              "2  spam  Free entry in 2 a wkly comp to win FA Cup fina...\n",
              "3   ham  U dun say so early hor... U c already then say...\n",
              "4   ham  Nah I don't think he goes to usf, he lives aro..."
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-b1d439bc-3ee8-4b7e-a709-d15d4c3380b6\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Label</th>\n",
              "      <th>EmailText</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>ham</td>\n",
              "      <td>Go until jurong point, crazy.. Available only ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>ham</td>\n",
              "      <td>Ok lar... Joking wif u oni...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>spam</td>\n",
              "      <td>Free entry in 2 a wkly comp to win FA Cup fina...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>ham</td>\n",
              "      <td>U dun say so early hor... U c already then say...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>ham</td>\n",
              "      <td>Nah I don't think he goes to usf, he lives aro...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-b1d439bc-3ee8-4b7e-a709-d15d4c3380b6')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-b1d439bc-3ee8-4b7e-a709-d15d4c3380b6 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-b1d439bc-3ee8-4b7e-a709-d15d4c3380b6');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-23fa9d3f-45d8-4472-b8cc-eeff95eb2c80\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-23fa9d3f-45d8-4472-b8cc-eeff95eb2c80')\"\n",
              "            title=\"Suggest charts.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-23fa9d3f-45d8-4472-b8cc-eeff95eb2c80 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "    </div>\n",
              "  </div>\n"
            ]
          },
          "metadata": {},
          "execution_count": 2
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "###### **Data-Preprocessing**\n"
      ],
      "metadata": {
        "id": "FyMCoPHK1Ve5"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Remove duplicate values**\n",
        "\n"
      ],
      "metadata": {
        "id": "22Qz09461biN"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "df = df.drop_duplicates(keep='first')\n"
      ],
      "metadata": {
        "id": "yBBVIKrJ1i2l"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Split into Independent and dependent variable**\n",
        "\n"
      ],
      "metadata": {
        "id": "oG9UQkkz1sd2"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "x = df['EmailText'].values\n",
        "y = df['Label'].values"
      ],
      "metadata": {
        "id": "H-cNdS1S1urD"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "###### **Text Pre-Processing**\n"
      ],
      "metadata": {
        "id": "415ee-Uo13xh"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "creating a function to lowercase the text, remove special characters, normalize certain words, use stems of words instead of the original form using porter stemmer algorithm."
      ],
      "metadata": {
        "id": "Os1mR9mx1_qT"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "porter_stemmer=PorterStemmer()\n",
        "def preprocessor(text):\n",
        "\n",
        "    text=text.lower()\n",
        "    text=re.sub(\"\\\\W\",\" \",text)\n",
        "    text=re.sub(\"\\\\s+(in|the|all|for|and|on)\\\\s+\",\" _connector_ \",text)\n",
        "    words=re.split(\"\\\\s+\",text)\n",
        "    stemmed_words=[porter_stemmer.stem(word=word) for word in words]\n",
        "    return ' '.join(stemmed_words)"
      ],
      "metadata": {
        "id": "Qfv-wBm72Kms"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "creating tokenizer function to\n",
        "\n",
        "Create a space between special characters\n",
        "\n",
        "Split based on whitespace"
      ],
      "metadata": {
        "id": "Xc6YTdbP2PlC"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def tokenizer(text):\n",
        "    text=re.sub(\"(\\\\W)\",\" \\\\1 \",text)\n",
        "    return re.split(\"\\\\s+\",text)"
      ],
      "metadata": {
        "id": "ad8FCpAV2ekN"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Count Vectorizer is used to transform a corpus of text to a vector of term.\n",
        "\n",
        "min_df = 0.06 ( taking 0.6% of Minimum Document Frequency )\n",
        "\n",
        "ngram_range=(1,2) ( word level Unigrams and bigrams)"
      ],
      "metadata": {
        "id": "qNbQ-Kdh2itP"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "vectorizer = CountVectorizer(tokenizer=tokenizer,ngram_range=(1,2),min_df=0.006,preprocessor=preprocessor)\n",
        "x  = vectorizer.fit_transform(x)"
      ],
      "metadata": {
        "id": "EcizO3ca2rrh",
        "outputId": "2b946951-6b6a-4db1-ffc4-a33577ea93f0",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/sklearn/feature_extraction/text.py:528: UserWarning: The parameter 'token_pattern' will not be used since 'tokenizer' is not None'\n",
            "  warnings.warn(\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "###### **Check Data is imbalanced**\n"
      ],
      "metadata": {
        "id": "3XBOc0oM2yyn"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "sns.countplot(df['Label'])\n"
      ],
      "metadata": {
        "id": "XVS8H-dZ25Au",
        "outputId": "6603712b-2b52-45f2-f209-194470b454f7",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 322
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "error",
          "ename": "ValueError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mValueError\u001b[0m                                Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-8-f1a230f43530>\u001b[0m in \u001b[0;36m<cell line: 1>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0msns\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcountplot\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdf\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m'Label'\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/seaborn/categorical.py\u001b[0m in \u001b[0;36mcountplot\u001b[0;34m(data, x, y, hue, order, hue_order, orient, color, palette, saturation, width, dodge, ax, **kwargs)\u001b[0m\n\u001b[1;32m   2941\u001b[0m         \u001b[0;32mraise\u001b[0m \u001b[0mValueError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"Cannot pass values for both `x` and `y`\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2942\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2943\u001b[0;31m     plotter = _CountPlotter(\n\u001b[0m\u001b[1;32m   2944\u001b[0m         \u001b[0mx\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0my\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mhue\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdata\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0morder\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mhue_order\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2945\u001b[0m         \u001b[0mestimator\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0merrorbar\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mn_boot\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0munits\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mseed\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/seaborn/categorical.py\u001b[0m in \u001b[0;36m__init__\u001b[0;34m(self, x, y, hue, data, order, hue_order, estimator, errorbar, n_boot, units, seed, orient, color, palette, saturation, width, errcolor, errwidth, capsize, dodge)\u001b[0m\n\u001b[1;32m   1528\u001b[0m                  errcolor, errwidth, capsize, dodge):\n\u001b[1;32m   1529\u001b[0m         \u001b[0;34m\"\"\"Initialize the plotter.\"\"\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1530\u001b[0;31m         self.establish_variables(x, y, hue, data, orient,\n\u001b[0m\u001b[1;32m   1531\u001b[0m                                  order, hue_order, units)\n\u001b[1;32m   1532\u001b[0m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mestablish_colors\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mcolor\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mpalette\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0msaturation\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/seaborn/categorical.py\u001b[0m in \u001b[0;36mestablish_variables\u001b[0;34m(self, x, y, hue, data, orient, order, hue_order, units)\u001b[0m\n\u001b[1;32m    514\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    515\u001b[0m                 \u001b[0;31m# Convert to a list of arrays, the common representation\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 516\u001b[0;31m                 \u001b[0mplot_data\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0masarray\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0md\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfloat\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0md\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mplot_data\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    517\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    518\u001b[0m                 \u001b[0;31m# The group names will just be numeric indices\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/seaborn/categorical.py\u001b[0m in \u001b[0;36m<listcomp>\u001b[0;34m(.0)\u001b[0m\n\u001b[1;32m    514\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    515\u001b[0m                 \u001b[0;31m# Convert to a list of arrays, the common representation\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 516\u001b[0;31m                 \u001b[0mplot_data\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m[\u001b[0m\u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0masarray\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0md\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mfloat\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mfor\u001b[0m \u001b[0md\u001b[0m \u001b[0;32min\u001b[0m \u001b[0mplot_data\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    517\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    518\u001b[0m                 \u001b[0;31m# The group names will just be numeric indices\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/pandas/core/series.py\u001b[0m in \u001b[0;36m__array__\u001b[0;34m(self, dtype)\u001b[0m\n\u001b[1;32m    891\u001b[0m               dtype='datetime64[ns]')\n\u001b[1;32m    892\u001b[0m         \"\"\"\n\u001b[0;32m--> 893\u001b[0;31m         \u001b[0;32mreturn\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0masarray\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_values\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdtype\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    894\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    895\u001b[0m     \u001b[0;31m# ----------------------------------------------------------------------\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mValueError\u001b[0m: could not convert string to float: 'ham'"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "target class has an uneven distribution of observations, So we are using random over sampling method to balance the target variable’s observation\n",
        "\n",
        "Random Oversampling: randomly duplicate examples in the minority class (Spam)"
      ],
      "metadata": {
        "id": "F1dvkxjB3Qi2"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from imblearn.under_sampling import NearMiss\n",
        "from collections import Counter\n",
        "from imblearn.over_sampling import RandomOverSampler\n",
        "\n",
        "ros = RandomOverSampler(random_state=42)\n",
        "\n",
        "print('Original dataset shape', Counter(y))\n",
        "\n",
        "# fit predictor and target\n",
        "x,y = ros.fit_resample(x, y)\n",
        "\n",
        "print('Modified dataset shape', Counter(y))"
      ],
      "metadata": {
        "id": "UGAFX75Y3PJb",
        "outputId": "3e52dae5-475b-4607-ec57-dcb29819b864",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Original dataset shape Counter({'ham': 4516, 'spam': 653})\n",
            "Modified dataset shape Counter({'ham': 4516, 'spam': 4516})\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "###### **Split into training and testing set**\n",
        "\n"
      ],
      "metadata": {
        "id": "e-XikCi13ZuX"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "x_train , x_test , y_train , y_test   = train_test_split(x, y, test_size =0.2,random_state = 0)"
      ],
      "metadata": {
        "id": "0ZxA9ml63h79"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "###### **Build SVM Model**\n"
      ],
      "metadata": {
        "id": "jrw0D3773nge"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "model   = SVC(C =1,kernel = \"linear\" )\n",
        "model.fit(x_train,y_train)"
      ],
      "metadata": {
        "id": "94TJdcEs3wst",
        "outputId": "c204ba36-8cc5-4c42-b8ea-bfa3d19f3c0b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 75
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "SVC(C=1, kernel='linear')"
            ],
            "text/html": [
              "<style>#sk-container-id-1 {color: black;background-color: white;}#sk-container-id-1 pre{padding: 0;}#sk-container-id-1 div.sk-toggleable {background-color: white;}#sk-container-id-1 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-1 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-1 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-1 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-1 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-1 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-1 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-1 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-1 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-1 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-1 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-1 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-1 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-1 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-1 div.sk-item {position: relative;z-index: 1;}#sk-container-id-1 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-1 div.sk-item::before, #sk-container-id-1 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-1 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-1 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-1 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-1 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-1 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-1 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-1 div.sk-label-container {text-align: center;}#sk-container-id-1 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-1 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-1\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>SVC(C=1, kernel=&#x27;linear&#x27;)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-1\" type=\"checkbox\" checked><label for=\"sk-estimator-id-1\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">SVC</label><div class=\"sk-toggleable__content\"><pre>SVC(C=1, kernel=&#x27;linear&#x27;)</pre></div></div></div></div></div>"
            ]
          },
          "metadata": {},
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Check accuracy score**\n",
        "\n"
      ],
      "metadata": {
        "id": "7ouBE5qA36Kb"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "accuracy = metrics.accuracy_score(y_test, model.predict(x_test))\n",
        "accuracy_percentage = 100 * accuracy\n",
        "accuracy_percentage"
      ],
      "metadata": {
        "id": "6rw6oLwJ3-gs",
        "outputId": "97f86bbc-3d1a-4f7b-cf9a-fbc57f6df12d",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "98.83785279468734"
            ]
          },
          "metadata": {},
          "execution_count": 12
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "###### **Hyperparameter Optimization using Grid Search CV**\n"
      ],
      "metadata": {
        "id": "iTCJ1q0d4DWN"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "params  = {\"C\":[0.2,0.5] , \"kernel\" : ['linear', 'sigmoid'] }\n"
      ],
      "metadata": {
        "id": "dSGgRaJE4Lo2"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "cval = KFold(n_splits = 2)\n",
        "model =  SVC();\n",
        "TunedModel = GridSearchCV(model,params,cv= cval)\n",
        "TunedModel.fit(x_train,y_train)"
      ],
      "metadata": {
        "id": "d4dwTeuI4RcJ",
        "outputId": "d40b291f-318f-4ded-cdd5-27bc6294e2df",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 118
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "GridSearchCV(cv=KFold(n_splits=2, random_state=None, shuffle=False),\n",
              "             estimator=SVC(),\n",
              "             param_grid={'C': [0.2, 0.5], 'kernel': ['linear', 'sigmoid']})"
            ],
            "text/html": [
              "<style>#sk-container-id-2 {color: black;background-color: white;}#sk-container-id-2 pre{padding: 0;}#sk-container-id-2 div.sk-toggleable {background-color: white;}#sk-container-id-2 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-2 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-2 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-2 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-2 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-2 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-2 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-2 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-2 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-2 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-2 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-2 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-2 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-2 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-2 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-2 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-2 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-2 div.sk-item {position: relative;z-index: 1;}#sk-container-id-2 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-2 div.sk-item::before, #sk-container-id-2 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-2 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-2 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-2 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-2 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-2 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-2 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-2 div.sk-label-container {text-align: center;}#sk-container-id-2 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-2 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-2\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>GridSearchCV(cv=KFold(n_splits=2, random_state=None, shuffle=False),\n",
              "             estimator=SVC(),\n",
              "             param_grid={&#x27;C&#x27;: [0.2, 0.5], &#x27;kernel&#x27;: [&#x27;linear&#x27;, &#x27;sigmoid&#x27;]})</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item sk-dashed-wrapped\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-2\" type=\"checkbox\" ><label for=\"sk-estimator-id-2\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">GridSearchCV</label><div class=\"sk-toggleable__content\"><pre>GridSearchCV(cv=KFold(n_splits=2, random_state=None, shuffle=False),\n",
              "             estimator=SVC(),\n",
              "             param_grid={&#x27;C&#x27;: [0.2, 0.5], &#x27;kernel&#x27;: [&#x27;linear&#x27;, &#x27;sigmoid&#x27;]})</pre></div></div></div><div class=\"sk-parallel\"><div class=\"sk-parallel-item\"><div class=\"sk-item\"><div class=\"sk-label-container\"><div class=\"sk-label sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-3\" type=\"checkbox\" ><label for=\"sk-estimator-id-3\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">estimator: SVC</label><div class=\"sk-toggleable__content\"><pre>SVC()</pre></div></div></div><div class=\"sk-serial\"><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-4\" type=\"checkbox\" ><label for=\"sk-estimator-id-4\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">SVC</label><div class=\"sk-toggleable__content\"><pre>SVC()</pre></div></div></div></div></div></div></div></div></div></div>"
            ]
          },
          "metadata": {},
          "execution_count": 14
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Accuracy of tuned model\n",
        "\n"
      ],
      "metadata": {
        "id": "iX0k1Kh24WVD"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "accuracy = metrics.accuracy_score(y_test, TunedModel.predict(x_test))\n",
        "accuracy_percentage = 100 * accuracy\n",
        "accuracy_percentage"
      ],
      "metadata": {
        "id": "JBR3oyE54Zf0",
        "outputId": "e2f8a5c3-26e1-499f-b0d0-3f643d7e380c",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "99.0038738240177"
            ]
          },
          "metadata": {},
          "execution_count": 15
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "###### **Model Evaluation**"
      ],
      "metadata": {
        "id": "uNih5A474l_N"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Confusion Matrix\n",
        "\n"
      ],
      "metadata": {
        "id": "895PiKk14yAm"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "sns.heatmap(confusion_matrix(y_test,TunedModel.predict(x_test)),annot = True , fmt =\"g\")\n",
        "plt.xlabel(\"Predicted\")\n",
        "plt.show(\"Actual\")\n",
        "plt.show()"
      ],
      "metadata": {
        "id": "AquVdwqe4zhN",
        "outputId": "98d82b28-d9ef-49bb-b60b-dcab78062d7e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 449
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 2 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgcAAAGwCAYAAAA9hsZrAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAApL0lEQVR4nO3df1hUZfrH8c8gMCICCsovTaOslDJ/UTTVZiaFZaVpmhu1uFlWoqmoJW26pRXlbr8sk/KrYalbWWlmpbHkj0pSw3TNlNQ0LQM1RAJjQOZ8/2gdOUdKaAeHmvera65LznnmmXvsUu/rvp/nOTbDMAwBAAD8l5+3AwAAAI0LyQEAADAhOQAAACYkBwAAwITkAAAAmJAcAAAAE5IDAABgQnIAAABM/L0dwDGVuz/zdghAoxPS8QZvhwA0Ss6KvQ06f9XBrz02V0CrMzw216lC5QAAAJg0msoBAACNhqva2xF4FckBAABWhsvbEXgVyQEAAFYu304OWHMAAABMqBwAAGBh0FYAAAAmtBUAAACOo3IAAIAVbQUAAGDi4+cc0FYAAAAmVA4AALCirQAAAEzYrQAAAHAclQMAACw4BAkAAJj5eFuB5AAAACsfrxyw5gAAAJhQOQAAwMrHD0EiOQAAwIq2AgAAwHFUDgAAsGK3AgAAMKGtAAAAcByVAwAArGgrAACAmgzDt7cy0lYAAAAmVA4AALDy8QWJJAcAAFix5gAAAJj4eOWANQcAAMCEygEAAFY8eAkAAJjQVgAAADiOygEAAFbsVgAAACa0FQAAAI6jcgAAgBVtBQAAYOLjyQFtBQAAYEJyAACAhWFUe+xVH9XV1Zo0aZLi4uIUFBSkM888U1OnTpVhGDViMzR58mTFxMQoKChISUlJ2r59u2me4uJipaSkKDQ0VC1atNCwYcNUVlZW5zhIDgAAsHK5PPeqh8cff1wzZ87Uc889p61bt+rxxx/XtGnT9Oyzz7rHTJs2TdOnT1dWVpbWrl2r4OBgJScnq6Kiwj0mJSVFW7ZsUU5OjpYuXarVq1dr+PDhdY7DZtRMR7yocvdn3g4BaHRCOt7g7RCARslZsbdB5/9pxf95bK6gXrfXeey1116rqKgozZ49231t4MCBCgoK0rx582QYhmJjYzVu3DiNHz9eknT48GFFRUUpOztbQ4YM0datWxUfH6/169crISFBkrRs2TJdc801+vbbbxUbG3vSOKgcAADQgJxOp0pLS00vp9NZ69iLL75Yubm5+uqrryRJmzZt0scff6yrr75akrRr1y4VFhYqKSnJ/Z6wsDAlJiYqLy9PkpSXl6cWLVq4EwNJSkpKkp+fn9auXVunmEkOAACw8mBbITMzU2FhYaZXZmZmrR87ceJEDRkyRB07dlRAQIC6deumMWPGKCUlRZJUWFgoSYqKijK9Lyoqyn2vsLBQkZGRpvv+/v4KDw93jzkZtjICAGDlwRMSMzIylJ6ebrpmt9trHfv6669r/vz5WrBggc4991xt3LhRY8aMUWxsrFJTUz0W08mQHAAA0IDsdvsvJgNWEyZMcFcPJKlz58765ptvlJmZqdTUVEVHR0uSioqKFBMT435fUVGRunbtKkmKjo7W/v37TfMePXpUxcXF7vefDG0FAACsvLRb4ciRI/LzM//T3KRJE7n+O09cXJyio6OVm5vrvl9aWqq1a9fK4XBIkhwOh0pKSpSfn+8e8+GHH8rlcikxMbFOcVA5AADAyksPXrruuuv0yCOPqF27djr33HP1+eef68knn9Rtt90mSbLZbBozZowefvhhnXXWWYqLi9OkSZMUGxur/v37S5I6deqkPn366I477lBWVpaqqqo0cuRIDRkypE47FSSSAwAAGo1nn31WkyZN0ogRI7R//37Fxsbqzjvv1OTJk91j7r33XpWXl2v48OEqKSnRpZdeqmXLlqlp06buMfPnz9fIkSPVu3dv+fn5aeDAgZo+fXqd4+CcA6AR45wDoHYNfs7B+3X/h/Rkgq6+x2NznSpUDgAAsOLBSwAAAMdROQAAwMpLCxIbC5IDAACsfLytQHIAAICVj1cOWHMAAABMqBwAAGBFWwEAAJjQVgAAADiOygEAAFa0FQAAgImPJwe0FQAAgAmVAwAArBrHMwm9huQAAAAr2goAAADHUTkAAMDKxysHJAcAAFj5+CFIJAcAAFj5eOWANQcAAMCEygEAAFZsZQQAACa0FQAAAI6jcgAAgJWPVw5IDgAAsPLxrYy0FQAAgAmVAwAALAwXuxUAAEBNPr7mgLYCAAAwoXIAAICVjy9IJDkAAMCKNQcAAMCENQcAAADHUTkAAMDKxysHJAcAAFj5+FMZaSsAAAATKgc+oLrapefnval3cz/RwUMlah3RUv2uvEx33txfNptNknTkpwo9NftVfZj3mQ6XlqlNdGul9EvW4GuT3PM4Kyv1jxfna9nKT1VZVaVLepyvv436q1q1DPPWVwM86tJLE5U+9k5163a+YmOjNGjQ7VryznL3/VmzntRfbh1kes8HH6zUddffeqpDRUOjrYA/ujmvv6PXl/5bj4y/S2e2b6st27/WpCdeVEhwkFL695EkTXthntZt/FKP3TtCsVGttWbDZj3y7EtqHdFSvRw9fh6TNU+r123UEw/co+bBzfTojGyNnfKUXnnqQS9+O8BzgpsF6T+btyp77uta+PqsWscsX75Cdwwf5/7Z6aw8VeHhVGIrI/7oNn75lXo5euiyxG6SpDbRrfX+ijxtLvjaPWbTl9t1/ZV/0gVd4iVJg665QgvfzdXmgp3q5eihH8uP6K3lK/X4xDQldj1XkjQ1/U71u2OCNm3dri6dzjr1XwzwsOUfrNTyD1b+6hins1JFRQdOTUCAl7DmwAd0jT9bazdu0e5vv5ckFez8Rhu2FOjSC7q4x3SJP0srP92gooPFMgxD6zZu0TffFeriHp0lSV9u36WjR6t1Ubfz3O85o12sYiIjtGnrjlP7hQAvuuyyi7R3z+fa/J+Venb6owoPb+HtkNAQDJfnXr9D9a4cHDx4UHPmzFFeXp4KCwslSdHR0br44os1dOhQtW7d2uNB4n8z7KbrVHbkJ11/+wQ18fNTtcule4YO0rVXXOIec/+IVD30zGwlpYySf5MmsvnZ9ODo25XQuZMk6WBxiQIC/BXaPNg0d0SLMB0sLjmVXwfwmg8+WKm3F7+vXbv36swz2mvKlHu15O1XdFnPfnL5eI/6D4e2Qt2tX79eycnJatasmZKSknT22WdLkoqKijR9+nQ99thjWr58uRISEn51HqfTKafTabpmc1bKbg+sZ/ioi+Wr1+rdDz/R4xPTdGb7NirY+Y0ez5rnXpgoSQve/kD/2bZDzz40TjGRrZS/eZsemZGt1hEt5eh+3kk+AfANCxcucf96y5Zt2vzFVm3b+ol69nRoxYpPvBgZ4Fn1Sg5GjRqlQYMGKSsry73K/RjDMHTXXXdp1KhRysvL+9V5MjMz9dBDD5muPTD6Dk0aM7w+4aCOnpi1QMNuuk5XX+6QJJ0d10779h/U/726RP2uvEwVzko9k/2anpk81r0u4Zwz2qng628094135eh+nlqFt1BV1VGVlpWbqgc/lBxWK8qq8FG7du3RgQM/6MwzTyc5+IMxfLwSVK/kYNOmTcrOzj4hMZAkm82msWPHqlu3biedJyMjQ+np6eb3f/9FfUJBPVQ4K+VnMy8vaeLnJ+O/h3wcPXpUR49Wy+Zn/v/q5+cn13/7ZfFnxcnfv4nWfr5FV/7pQknSrr379P3+H9SlU4dT8C2AxqdNm2hFRLRU4ff7vR0KPI22Qt1FR0dr3bp16tixY633161bp6ioqJPOY7fbZbfbTdcqi2kpNJSeF3XTi68uVkxkhM5s31bbdu7Wy2+9r/5X9ZQkNQ9upoTzO+nJWf9S08BAxUS10mf/2ap3/v2RJgy/RZIUEtxMA5Iv1z9enKewkGAFBzdT5oy56tLpLHYq4A8jOLiZzjzzdPfPp59+ms4/P16HDpWouLhED/xtrBYtfk9FRQd0xhnt9egj92vnzt36IGeV94JGw/idLiT0FJth1P2MyBkzZmjcuHG688471bt3b3ciUFRUpNzcXM2aNUv//Oc/NWLEiHoHUrn7s3q/B3VTfuQnPTf3DeWuWa/iklK1jmipqy936O6UAQoI+Dk/PFhcoqfnvKa8DZt1+McyxUS20o3XXKG/DLjaXSk6dgjS+yvyVFV1VBcndNYDI/9KW6EBhXS8wdsh+JTLLrtIOR8sPOH6y68s1KhR9+uNhf+nLl3OVYsWodr3fZFy/71aDz70T+3ff9AL0fo2Z8XeBp2//OFbPDZX8APzPDbXqVKv5ECSXnvtNT311FPKz89XdXW1JKlJkybq0aOH0tPTNXjw4N8UCMkBcCKSA6B2DZ4cTEnx2FzBk+d7bK5Tpd5bGW+66SbddNNNqqqq0sGDP2fLrVq1UkBAgMeDAwDAK1iQ+NsEBAQoJibGk7EAAIBGgOOTAQCwYrcCAAAw8fHdCjxbAQAAmFA5AADAirYCAACoydePT6atAAAATKgcAABgRVsBAACYkBwAAAATtjICAAAcR+UAAAAr2goAAKAmw8eTA9oKAADAhMoBAABWPl45IDkAAMCKExIBAACOo3IAAIAVbQUAAGDi48kBbQUAAGBC5QAAAAvD8O3KAckBAABWPt5WIDkAAMDKx5MD1hwAAAATkgMAACwMl+GxV3199913uuWWWxQREaGgoCB17txZn3322fHYDEOTJ09WTEyMgoKClJSUpO3bt5vmKC4uVkpKikJDQ9WiRQsNGzZMZWVldY6B5AAAACuX4blXPRw6dEiXXHKJAgIC9P777+vLL7/UE088oZYtW7rHTJs2TdOnT1dWVpbWrl2r4OBgJScnq6Kiwj0mJSVFW7ZsUU5OjpYuXarVq1dr+PDhdY7DZjSSJZmVuz87+SDAx4R0vMHbIQCNkrNib4POfzi1t8fmavrie3I6naZrdrtddrv9hLETJ07UJ598oo8++qjWuQzDUGxsrMaNG6fx48f/HOvhw4qKilJ2draGDBmirVu3Kj4+XuvXr1dCQoIkadmyZbrmmmv07bffKjY29qQxUzkAAMDK5blXZmamwsLCTK/MzMxaP3bJkiVKSEjQoEGDFBkZqW7dumnWrFnu+7t27VJhYaGSkpLc18LCwpSYmKi8vDxJUl5enlq0aOFODCQpKSlJfn5+Wrt2bZ2+PrsVAACw+C1rBX5JRkaG0tPTTddqqxpI0tdff62ZM2cqPT1d999/v9avX6977rlHgYGBSk1NVWFhoSQpKirK9L6oqCj3vcLCQkVGRpru+/v7Kzw83D3mZEgOAABoQL/UQqiNy+VSQkKCHn30UUlSt27d9MUXXygrK0upqakNGaYJbQUAAKy8tCAxJiZG8fHxpmudOnXSnj17JEnR0dGSpKKiItOYoqIi973o6Gjt37/fdP/o0aMqLi52jzkZkgMAAKw8uOagPi655BIVFBSYrn311Vdq3769JCkuLk7R0dHKzc113y8tLdXatWvlcDgkSQ6HQyUlJcrPz3eP+fDDD+VyuZSYmFinOGgrAADQSIwdO1YXX3yxHn30UQ0ePFjr1q3Tiy++qBdffFGSZLPZNGbMGD388MM666yzFBcXp0mTJik2Nlb9+/eX9HOloU+fPrrjjjuUlZWlqqoqjRw5UkOGDKnTTgWJ5AAAgBN4ckFifVxwwQVatGiRMjIyNGXKFMXFxenpp59WSkqKe8y9996r8vJyDR8+XCUlJbr00ku1bNkyNW3a1D1m/vz5GjlypHr37i0/Pz8NHDhQ06dPr3McnHMANGKccwDUrqHPOTg08HKPzdXyzZUem+tUoXIAAICFtyoHjQULEgEAgAmVAwAArOq5y+CPhuQAAAALw8eTA9oKAADAhMoBAABWPl45IDkAAMCCtgIAAEANVA4AALDy8coByQEAABa+3lYgOQAAwMLXkwPWHAAAABMqBwAAWPh65YDkAAAAK8Pm7Qi8irYCAAAwoXIAAIAFbQUAAGBiuGgrAAAAuFE5AADAgrYCAAAwMditAAAAcByVAwAALGgrAAAAE1/frUByAACAhWF4OwLvYs0BAAAwoXIAAIAFbQUAAGDi68kBbQUAAGBC5QAAAAtfX5BIcgAAgAVtBQAAgBqoHAAAYOHrz1YgOQAAwMLXj0+mrQAAAEyoHAAAYOGirQAAAGpizQEAADBhKyMAAEANVA4AALDghEQAAGBCWwEAAKAGKgcAAFiwlREAAJj4+lZG2goAAMCEygEAABbsVgAAACa+vuaAtgIAADChcgAAgIWvL0gkOQAAwII1BwAAwIQ1BwAAADU0mspBs7P7eTsEoNH5ad9H3g4B8EmsOQAAACa0FQAAAGqgcgAAgIWPb1YgOQAAwIq2AgAAQA1UDgAAsGC3AgAAMHF5OwAvo60AAABMqBwAAGBhiLYCAACoweXjexlJDgAAsHD5eOWANQcAAMCEygEAABasOQAAACZsZQQAAKiBygEAABa0FQAAgAltBQAAgBpIDgAAsHB58PVbPfbYY7LZbBozZoz7WkVFhdLS0hQREaHmzZtr4MCBKioqMr1vz5496tu3r5o1a6bIyEhNmDBBR48erddnkxwAAGBhyOax12+xfv16vfDCCzr//PNN18eOHat33nlHCxcu1KpVq7Rv3z4NGDDAfb+6ulp9+/ZVZWWl1qxZo7lz5yo7O1uTJ0+u1+eTHAAA0IiUlZUpJSVFs2bNUsuWLd3XDx8+rNmzZ+vJJ5/UFVdcoR49euill17SmjVr9Omnn0qSPvjgA3355ZeaN2+eunbtqquvvlpTp07VjBkzVFlZWecYSA4AALBw2Tz3cjqdKi0tNb2cTucvfnZaWpr69u2rpKQk0/X8/HxVVVWZrnfs2FHt2rVTXl6eJCkvL0+dO3dWVFSUe0xycrJKS0u1ZcuWOn9/kgMAACxcsnnslZmZqbCwMNMrMzOz1s999dVXtWHDhlrvFxYWKjAwUC1atDBdj4qKUmFhoXtMzcTg2P1j9+qKrYwAAFh48qGMGRkZSk9PN12z2+0njNu7d69Gjx6tnJwcNW3a1IMR1B+VAwAAGpDdbldoaKjpVVtykJ+fr/3796t79+7y9/eXv7+/Vq1apenTp8vf319RUVGqrKxUSUmJ6X1FRUWKjo6WJEVHR5+we+HYz8fG1AXJAQAAFt7Yyti7d29t3rxZGzdudL8SEhKUkpLi/nVAQIByc3Pd7ykoKNCePXvkcDgkSQ6HQ5s3b9b+/fvdY3JychQaGqr4+Pg6x0JbAQAAC5ft1B+fHBISovPOO890LTg4WBEREe7rw4YNU3p6usLDwxUaGqpRo0bJ4XDooosukiRdddVVio+P16233qpp06apsLBQDzzwgNLS0mqtVvwSkgMAAH4nnnrqKfn5+WngwIFyOp1KTk7W888/777fpEkTLV26VHfffbccDoeCg4OVmpqqKVOm1OtzbIZheHLdxW/mH9jG2yEAjc5P+z7ydghAoxTQ6owGnX9hTIrH5hr0/XyPzXWqUDkAAMCCBy8BAADUQOUAAAAL16lfj9iokBwAAGDh+o0PTPqjoK0AAABMqBwAAGDRKLbxeRHJAQAAFqw5AAAAJmxlBAAAqIHKAQAAFqw5AAAAJr6+5oC2AgAAMKFyAACAha8vSCQ5AADAwteTA9oKAADAhMoBAAAWho8vSCQ5AADAgrYCAABADVQOAACw8PXKAckBAAAWnJAIAABMOCERAACgBioHAABYsOYAAACY+HpyQFsBAACYUDkAAMCC3QoAAMCE3QoAAAA1UDkAAMDC1xckkhwAAGDh62sOaCsAAAATKgcAAFi4fLx2QHIAAIAFaw4AAICJb9cNWHMAAAAsqBwAAGBBWwEAAJhwQiIAAEANVA4AALBgKyMAADDx7dSAtgIAALCgcgAAgAW7FQAAgImvrzmgrQAAAEyoHAAAYOHbdQOSAwAATsCaAwAAYMKaAwAAgBqoHAAAYOHbdQOSAwAATuDraw5oKwAAABMqBwAAWBg+3lggOQAAwIK2AgAAQA1UDgAAsPD1cw5IDgAAsPDt1IC2AgAAsCA5gCTpzuF/0Yb8HBUf3Kbig9v08eol6pPcy9thAQ2qvPyIHns6S1cOSFWPXv2Ucme6Nm8tcN83DEPPzXpZl19/s3r06qfbR2fom73fmeb4smCHbh99vxzJN+qSqwfrwcef0ZEjP53qrwIPc8nw2Ov3iOQAkqTvvvtef/tbpi686GolOq7RipWf6K035yg+/mxvhwY0mMmPPaO89Z8rc/J4LXplpi6+sLvuGH2/ig4clCTNmb9Q899YoskTRmnBrKcV1LSp7kx/QE5npSRp/4EfdPvoDLVrG6MFLz6trCenaseuPfrbI09482vBA1wefP0ekRxAkrT03Ry9v+xD7dixS9u3f61Jkx9XWVm5Ei/s7u3QgAZR4XTq36s+VnraMCV07ax2bWOVNuwWtWsbq9cWvSvDMPTK64s1PHWIrviTQ+d0iNOjk8Zr/8EflPvRGknSqjVr5e/vrwfGpSmufVt17nSOJk8YqZyVn2jPt/u8/A3xvzA8+N/vEckBTuDn56fBg69XcHAzfbo239vhAA2i+mi1qqtdsgcGmK7b7YHa8J8t+nZfoQ7+cEiOhG7ueyHNg3V+/Dna9MU2SVJlZZUCAvzl53f8r9KmdrskacOmLafgWwANwyvJgdPpVGlpqellGL/P7OqP5LzzOqqk+CsdKdul5597TDcOul1bt273dlhAgwgObqYu53VSVva/tP/AD6qurtY7yz/Upi+26eDBYh0sPiRJighvaXpfRHhLHfzh53uJPbrqhx8Oac78N1RVVaXDpT/qqZlzJEkHfig+tV8IHkVbwcP27t2r22677VfHZGZmKiwszPQyXD96OhTUU0HBTvW44CpdfMm1euHFlzVn9tPq1Oksb4cFNJjMSeMlw9AV/W9R917Xa/7Ct3V1Uk/Z/Or2V2OHM9rrkQfGae6rbymhd39dfv3NahMTrYjwlvLzszVw9GhIvt5W8Pg5B8XFxZo7d67mzJnzi2MyMjKUnp5uutYyoqOnQ0E9VVVVaefO3ZKkDZ9vVkKPrho18naNSLvPu4EBDaRd21hlz/iHjvxUofLyI2rdKlzjJmWqbWy0Wv23YvBD8SG1bhXufs8PxYd0zllnun/ue1Uv9b2qlw4WH1Kzpk0lm00vv7ZIbWNjTvn3ATyl3snBkiVLfvX+119/fdI57Ha77P/tyx1js5FlNzZ+fn6y2wO9HQbQ4JoFNVWzoKY6XPqj1qzLV/qI235OECJa6tP8jep49s/JQFl5uf7zZYEG39D3hDmOJRNvLV0ue2CAHBd0O2EMfj9+r+0AT6l3ctC/f3/ZbLZfXSPAP/S/P488PFHLlq3Qnr3fKSSkuf48pL969nTomr43ezs0oMF8sjZfhmHo9HZttefbfXpixmzFtWur/n2vks1m062D++vFua+qfds2ahMbpedmvaLIVhHq/aeL3XMseGOJunaOV7Ogpspb/7memDFbY+7+q0JDmnvxm+F/5fLxdXD1Tg5iYmL0/PPPq1+/frXe37hxo3r06PE/B4ZTq3XrVnppzjOKiYnU4cM/avPmrbqm7836d+5H3g4NaDA/lpXr6ayXVHTgoMJCQ3Rlz0t1z52pCvD/+a/G21IG6aefKvTgtOn6saxM3c8/V1lPTDVV1DZv/UozZs/TkZ9+Ulz70zT53lG6vk9vb30lwCNsRj23CVx//fXq2rWrpkyZUuv9TZs2qVu3bnK56leU8Q9sU6/xgC/4aR/JGVCbgFZnNOj8t7Qf4LG55n3zlsfmOlXqXTmYMGGCysvLf/F+hw4dtGLFiv8pKAAAvOn3euyxp9Q7OfjTn/70q/eDg4PVs2fP3xwQAADwLh7ZDACAxe/1fAJP4fhkAAAsvHVCYmZmpi644AKFhIQoMjJS/fv3V0FBgWlMRUWF0tLSFBERoebNm2vgwIEqKioyjdmzZ4/69u2rZs2aKTIyUhMmTNDRo0frHAfJAQAAFt56ZPOqVauUlpamTz/9VDk5OaqqqtJVV11lWus3duxYvfPOO1q4cKFWrVqlffv2acCA4wsoq6ur1bdvX1VWVmrNmjWaO3eusrOzNXny5DrHUe/dCg2F3QrAiditANSuoXcrDGpf+3b932LeV6/L6XSartV2GGBtDhw4oMjISK1atUqXXXaZDh8+rNatW2vBggW68cYbJUnbtm1Tp06dlJeXp4suukjvv/++rr32Wu3bt09RUVGSpKysLN133306cOCAAgNPfrgdlQMAACw8+WyF2p4nlJmZWac4Dh8+LEkKD//5CO/8/HxVVVUpKSnJPaZjx45q166d8vLyJEl5eXnq3LmzOzGQpOTkZJWWlmrLlro9LZQFiQAAWHjy+OTanidUl6qBy+XSmDFjdMkll+i8886TJBUWFiowMFAtWrQwjY2KilJhYaF7TM3E4Nj9Y/fqguQAAIAGVNcWglVaWpq++OILffzxxw0Q1a+jrQAAgIVhGB57/RYjR47U0qVLtWLFCrVt29Z9PTo6WpWVlSopKTGNLyoqUnR0tHuMdffCsZ+PjTkZkgMAACy8tVvBMAyNHDlSixYt0ocffqi4uDjT/R49eiggIEC5ubnuawUFBdqzZ48cDockyeFwaPPmzdq/f797TE5OjkJDQxUfH1+nOGgrAADQSKSlpWnBggV6++23FRIS4l4jEBYWpqCgIIWFhWnYsGFKT09XeHi4QkNDNWrUKDkcDl100UWSpKuuukrx8fG69dZbNW3aNBUWFuqBBx5QWlpandsbJAcAAFh4ckFifcycOVOSdPnll5uuv/TSSxo6dKgk6amnnpKfn58GDhwop9Op5ORkPf/88+6xTZo00dKlS3X33XfL4XAoODhYqampv/jAxNpwzgHQiHHOAVC7hj7n4Np2fT0219I973psrlOFNQcAAMCEtgIAABY8shkAAJg0ko6715AcAABg4a0FiY0Faw4AAIAJlQMAACwM1hwAAICafH1BIm0FAABgQuUAAAALdisAAAAT2goAAAA1UDkAAMCC3QoAAMDE5eNrDmgrAAAAEyoHAABY+HbdgOQAAIAT+PpuBZIDAAAsfD05YM0BAAAwoXIAAIAFJyQCAAAT2goAAAA1UDkAAMCCExIBAICJr685oK0AAABMqBwAAGDh6wsSSQ4AALCgrQAAAFADlQMAACxoKwAAABO2MgIAABMXaw4AAACOo3IAAIAFbQUAAGBCWwEAAKAGKgcAAFjQVgAAACa0FQAAAGqgcgAAgAVtBQAAYEJbAQAAoAYqBwAAWNBWAAAAJobh8nYIXkVyAACAha8/spk1BwAAwITKAQAAFoaP71YgOQAAwIK2AgAAQA1UDgAAsKCtAAAATDghEQAAoAYqBwAAWHBCIgAAMPH1NQe0FQAAgAmVAwAALHz9nAOSAwAALHy9rUByAACABVsZAQAAaqByAACABW0FAABg4usLEmkrAAAAEyoHAABY0FYAAAAm7FYAAACogcoBAAAWPHgJAACY0FYAAACogcoBAAAW7FYAAAAmrDkAAAAmvl45YM0BAAAwoXIAAICFr1cOSA4AALDw7dSAtgIAALCwGb5eO4GJ0+lUZmamMjIyZLfbvR0O0Cjw5wK+huQAJqWlpQoLC9Phw4cVGhrq7XCARoE/F/A1tBUAAIAJyQEAADAhOQAAACYkBzCx2+36+9//zqIroAb+XMDXsCARAACYUDkAAAAmJAcAAMCE5AAAAJiQHAAAABOSA7jNmDFDp59+upo2barExEStW7fO2yEBXrV69Wpdd911io2Nlc1m0+LFi70dEnBKkBxAkvTaa68pPT1df//737VhwwZ16dJFycnJ2r9/v7dDA7ymvLxcXbp00YwZM7wdCnBKsZURkqTExERdcMEFeu655yRJLpdLp512mkaNGqWJEyd6OTrA+2w2mxYtWqT+/ft7OxSgwVE5gCorK5Wfn6+kpCT3NT8/PyUlJSkvL8+LkQEAvIHkADp48KCqq6sVFRVluh4VFaXCwkIvRQUA8BaSAwAAYEJyALVq1UpNmjRRUVGR6XpRUZGio6O9FBUAwFtIDqDAwED16NFDubm57msul0u5ublyOBxejAwA4A3+3g4AjUN6erpSU1OVkJCgCy+8UE8//bTKy8v117/+1duhAV5TVlamHTt2uH/etWuXNm7cqPDwcLVr186LkQENi62McHvuuef0j3/8Q4WFherataumT5+uxMREb4cFeM3KlSvVq1evE66npqYqOzv71AcEnCIkBwAAwIQ1BwAAwITkAAAAmJAcAAAAE5IDAABgQnIAAABMSA4AAIAJyQEAADAhOQAAACYkB0AjMXToUPXv39/98+WXX64xY8ac8jhWrlwpm82mkpKSU/7ZABoHkgPgJIYOHSqbzSabzabAwEB16NBBU6ZM0dGjRxv0c9966y1NnTq1TmP5Bx2AJ/HgJaAO+vTpo5deeklOp1Pvvfee0tLSFBAQoIyMDNO4yspKBQYGeuQzw8PDPTIPANQXlQOgDux2u6Kjo9W+fXvdfffdSkpK0pIlS9ytgEceeUSxsbE655xzJEl79+7V4MGD1aJFC4WHh6tfv37avXu3e77q6mqlp6erRYsWioiI0L333ivrY06sbQWn06n77rtPp512mux2uzp06KDZs2dr9+7d7ocDtWzZUjabTUOHDpX086O3MzMzFRcXp6CgIHXp0kVvvPGG6XPee+89nX322QoKClKvXr1McQLwTSQHwG8QFBSkyspKSVJubq4KCgqUk5OjpUuXqqqqSsnJyQoJCdFHH32kTz75RM2bN1efPn3c73niiSeUnZ2tOXPm6OOPP1ZxcbEWLVr0q5/5l7/8Rf/61780ffp0bd26VS+88IKaN2+u0047TW+++aYkqaCgQN9//72eeeYZSVJmZqZefvllZWVlacuWLRo7dqxuueUWrVq1StLPScyAAQN03XXXaePGjbr99ts1ceLEhvptA/B7YQD4VampqUa/fv0MwzAMl8tl5OTkGHa73Rg/fryRmppqREVFGU6n0z3+lVdeMc455xzD5XK5rzmdTiMoKMhYvny5YRiGERMTY0ybNs19v6qqymjbtq37cwzDMHr27GmMHj3aMAzDKCgoMCQZOTk5tca4YsUKQ5Jx6NAh97WKigqjWbNmxpo1a0xjhw0bZvz5z382DMMwMjIyjPj4eNP9++6774S5APgW1hwAdbB06VI1b95cVVVVcrlcuvnmm/Xggw8qLS1NnTt3Nq0z2LRpk3bs2KGQkBDTHBUVFdq5c6cOHz6s77//XomJie57/v7+SkhIOKG1cMzGjRvVpEkT9ezZs84x79ixQ0eOHNGVV15pul5ZWalu3bpJkrZu3WqKQ5IcDkedPwPAHxPJAVAHvXr10syZMxUYGKjY2Fj5+x//oxMcHGwaW1ZWph49emj+/PknzNO6devf9PlBQUH1fk9ZWZkk6d1331WbNm1M9+x2+2+KA4BvIDkA6iA4OFgdOnSo09ju3bvrtddeU2RkpEJDQ2sdExMTo7Vr1+qyyy6TJB09elT5+fnq3r17reM7d+4sl8ulVatWKSkp6YT7xyoX1dXV7mvx8fGy2+3as2fPL1YcOnXqpCVLlpiuffrppyf/kgD+0FiQCHhYSkqKWrVqpX79+umjjz7Srl27tHLlSt1zzz369ttvJUmjR4/WY489psWLF2vbtm0aMWLEr55RcPrppys1NVW33XabFi9e7J7z9ddflyS1b99eNptNS5cu1YEDB1RWVqaQkBCNHz9eY8eO1dy5c7Vz505t2LBBzz77rObOnStJuuuuu7R9+3ZNmDBBBQUFWrBggbKzsxv6twhAI0dyAHhYs2bNtHr1arVr104DBgxQp06dNGzYMFVUVLgrCePGjdOtt96q1NRUORwOhYSE6IYbbvjVeWfOnKkbb7xRI0aMUMeOHXXHHXeovLxcktSmTRs99NBDmjhxoqKiojRy5EhJ0tSpUzVp0iRlZmaqU6dO6tOnj959913FxcVJktq1a6c333xTixcvVpcuXZSVlaVHH320AX93APwe2IxfWgEFAAB8EpUDAABgQnIAAABMSA4AAIAJyQEAADAhOQAAACYkBwAAwITkAAAAmJAcAAAAE5IDAABgQnIAAABMSA4AAIDJ/wOPBAMeIt/AXQAAAABJRU5ErkJggg==\n"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**classification report**"
      ],
      "metadata": {
        "id": "166FRmYf496G"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "print(classification_report(y_test,TunedModel.predict(x_test)))\n"
      ],
      "metadata": {
        "id": "zpVkymyH5AKf",
        "outputId": "55403108-99b9-42ee-acac-88d5405e1ab7",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "              precision    recall  f1-score   support\n",
            "\n",
            "         ham       1.00      0.98      0.99       895\n",
            "        spam       0.98      1.00      0.99       912\n",
            "\n",
            "    accuracy                           0.99      1807\n",
            "   macro avg       0.99      0.99      0.99      1807\n",
            "weighted avg       0.99      0.99      0.99      1807\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Using the trained model, predict whether the following five emails are spam or ham"
      ],
      "metadata": {
        "id": "wQIr9tP35HCh"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "mails = [\"Hey, you have won a car !!!!. Conrgratzz\"\n",
        "              ,\"Dear applicant, Your CV has been recieved. Best regards\"\n",
        "              ,\"You have received $1000000 to your account\"\n",
        "              ,\"Join with our whatsapp group\"\n",
        "              ,\"Kindly check the previous email. Kind Regard\"]\n",
        "for mail in mails:\n",
        "  is_spam = TunedModel.predict(vectorizer.transform([mail]).toarray())\n",
        "  print(mail + \" : \"+ is_spam)"
      ],
      "metadata": {
        "id": "YFlSyn0X5LKz",
        "outputId": "f3021622-c13e-4d4e-f080-96802757e210",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "['Hey, you have won a car !!!!. Conrgratzz : spam']\n",
            "['Dear applicant, Your CV has been recieved. Best regards : ham']\n",
            "['You have received $1000000 to your account : spam']\n",
            "['Join with our whatsapp group : ham']\n",
            "['Kindly check the previous email. Kind Regard : ham']\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### **1.2.6 -Emotion Detection [10]**"
      ],
      "metadata": {
        "id": "DK2uouG-Pb3X"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Choosing Your Dataset**"
      ],
      "metadata": {
        "id": "RdOo99ZxPvTu"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from google.colab import drive\n",
        "drive.mount('/content/drive')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "iYlqQbU2QEQK",
        "outputId": "6ed6f035-9cba-4b17-8c43-1dd010b31a91"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Mounted at /content/drive\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#%%\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "data = pd.read_csv('/content/drive/MyDrive/Courses /Data Science /NLP/Datasets/Emotion /text_emotion.csv')"
      ],
      "metadata": {
        "id": "uLeqaegZP5-K"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Understanding What’s Inside Your Dataset**"
      ],
      "metadata": {
        "id": "4LFLBAs2TLJe"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "data.head()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "ma0CRhdUTBoQ",
        "outputId": "5c936283-d0a9-4c0a-830a-2b88c5f8ffa0"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "     tweet_id   sentiment       author  \\\n",
              "0  1956967341       empty   xoshayzers   \n",
              "1  1956967666     sadness    wannamama   \n",
              "2  1956967696     sadness    coolfunky   \n",
              "3  1956967789  enthusiasm  czareaquino   \n",
              "4  1956968416     neutral    xkilljoyx   \n",
              "\n",
              "                                             content  \n",
              "0  @tiffanylue i know  i was listenin to bad habi...  \n",
              "1  Layin n bed with a headache  ughhhh...waitin o...  \n",
              "2                Funeral ceremony...gloomy friday...  \n",
              "3               wants to hang out with friends SOON!  \n",
              "4  @dannycastillo We want to trade with someone w...  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-aeffcd2f-c53d-4d8e-8f56-e03e5677f52f\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>tweet_id</th>\n",
              "      <th>sentiment</th>\n",
              "      <th>author</th>\n",
              "      <th>content</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1956967341</td>\n",
              "      <td>empty</td>\n",
              "      <td>xoshayzers</td>\n",
              "      <td>@tiffanylue i know  i was listenin to bad habi...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1956967666</td>\n",
              "      <td>sadness</td>\n",
              "      <td>wannamama</td>\n",
              "      <td>Layin n bed with a headache  ughhhh...waitin o...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>1956967696</td>\n",
              "      <td>sadness</td>\n",
              "      <td>coolfunky</td>\n",
              "      <td>Funeral ceremony...gloomy friday...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>1956967789</td>\n",
              "      <td>enthusiasm</td>\n",
              "      <td>czareaquino</td>\n",
              "      <td>wants to hang out with friends SOON!</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>1956968416</td>\n",
              "      <td>neutral</td>\n",
              "      <td>xkilljoyx</td>\n",
              "      <td>@dannycastillo We want to trade with someone w...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-aeffcd2f-c53d-4d8e-8f56-e03e5677f52f')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-aeffcd2f-c53d-4d8e-8f56-e03e5677f52f button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-aeffcd2f-c53d-4d8e-8f56-e03e5677f52f');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-e6307115-3e97-482c-8057-68fe0f84e4ec\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-e6307115-3e97-482c-8057-68fe0f84e4ec')\"\n",
              "            title=\"Suggest charts.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-e6307115-3e97-482c-8057-68fe0f84e4ec button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "    </div>\n",
              "  </div>\n"
            ]
          },
          "metadata": {},
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "This one’s a simple dataset with just four columns, the tweet ID, emotion depicted by the tweet, the author, and the text content of the tweet.\n",
        "We do not necessarily need the author column. Hence we can drop it."
      ],
      "metadata": {
        "id": "euKsiOFeTRXI"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "data = data.drop('author', axis=1)\n"
      ],
      "metadata": {
        "id": "t2rW6SoZTTnY"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "data.head()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "JhsSvxFqTYgN",
        "outputId": "b6f0b325-36e1-443b-cb93-399f73c12077"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "     tweet_id   sentiment                                            content\n",
              "0  1956967341       empty  @tiffanylue i know  i was listenin to bad habi...\n",
              "1  1956967666     sadness  Layin n bed with a headache  ughhhh...waitin o...\n",
              "2  1956967696     sadness                Funeral ceremony...gloomy friday...\n",
              "3  1956967789  enthusiasm               wants to hang out with friends SOON!\n",
              "4  1956968416     neutral  @dannycastillo We want to trade with someone w..."
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-4c97c72e-fa20-42be-9af0-6d430abb960b\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>tweet_id</th>\n",
              "      <th>sentiment</th>\n",
              "      <th>content</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1956967341</td>\n",
              "      <td>empty</td>\n",
              "      <td>@tiffanylue i know  i was listenin to bad habi...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1956967666</td>\n",
              "      <td>sadness</td>\n",
              "      <td>Layin n bed with a headache  ughhhh...waitin o...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>1956967696</td>\n",
              "      <td>sadness</td>\n",
              "      <td>Funeral ceremony...gloomy friday...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>1956967789</td>\n",
              "      <td>enthusiasm</td>\n",
              "      <td>wants to hang out with friends SOON!</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>1956968416</td>\n",
              "      <td>neutral</td>\n",
              "      <td>@dannycastillo We want to trade with someone w...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-4c97c72e-fa20-42be-9af0-6d430abb960b')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-4c97c72e-fa20-42be-9af0-6d430abb960b button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-4c97c72e-fa20-42be-9af0-6d430abb960b');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-12952bdb-8bb1-46ea-8cfe-b0214d69fc86\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-12952bdb-8bb1-46ea-8cfe-b0214d69fc86')\"\n",
              "            title=\"Suggest charts.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-12952bdb-8bb1-46ea-8cfe-b0214d69fc86 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "    </div>\n",
              "  </div>\n"
            ]
          },
          "metadata": {},
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "The dataset has 40,000 tweets in total, labeled into 13 different human sentiments. Our task here is to build a model, such that give a new tweet or text sentence, it can accurately identify which of the emotions (for which it is trained to recognize) it depicts.\n",
        "For this tutorial let us consider just two of these sentiments for simplicity, ‘happiness’ and ‘sadness’ (which constitute a total of about 10,000 tweets from the entire sample of data). We can thus drop rows with all other labels."
      ],
      "metadata": {
        "id": "Utf9LOaFToKS"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Dropping rows with other emotion labels\n",
        "data = data.drop(data[data.sentiment == 'anger'].index)\n",
        "data = data.drop(data[data.sentiment == 'boredom'].index)\n",
        "data = data.drop(data[data.sentiment == 'enthusiasm'].index)\n",
        "data = data.drop(data[data.sentiment == 'empty'].index)\n",
        "data = data.drop(data[data.sentiment == 'fun'].index)\n",
        "data = data.drop(data[data.sentiment == 'relief'].index)\n",
        "data = data.drop(data[data.sentiment == 'surprise'].index)\n",
        "data = data.drop(data[data.sentiment == 'love'].index)\n",
        "data = data.drop(data[data.sentiment == 'hate'].index)\n",
        "data = data.drop(data[data.sentiment == 'neutral'].index)\n",
        "data = data.drop(data[data.sentiment == 'worry'].index)"
      ],
      "metadata": {
        "id": "UK2i9hC_TqCe"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "data.head()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "DuvsK8XnTyPO",
        "outputId": "401d3771-2c90-4be4-e8ff-45ca4b6ab4d0"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "     tweet_id sentiment                                            content\n",
              "1  1956967666   sadness  Layin n bed with a headache  ughhhh...waitin o...\n",
              "2  1956967696   sadness                Funeral ceremony...gloomy friday...\n",
              "6  1956968487   sadness  I should be sleep, but im not! thinking about ...\n",
              "8  1956969035   sadness            @charviray Charlene my love. I miss you\n",
              "9  1956969172   sadness         @kelcouch I'm sorry  at least it's Friday?"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-82cadee5-1b2f-43dc-97f7-7ea6b7ad3401\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>tweet_id</th>\n",
              "      <th>sentiment</th>\n",
              "      <th>content</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1956967666</td>\n",
              "      <td>sadness</td>\n",
              "      <td>Layin n bed with a headache  ughhhh...waitin o...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>1956967696</td>\n",
              "      <td>sadness</td>\n",
              "      <td>Funeral ceremony...gloomy friday...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>1956968487</td>\n",
              "      <td>sadness</td>\n",
              "      <td>I should be sleep, but im not! thinking about ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>1956969035</td>\n",
              "      <td>sadness</td>\n",
              "      <td>@charviray Charlene my love. I miss you</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>9</th>\n",
              "      <td>1956969172</td>\n",
              "      <td>sadness</td>\n",
              "      <td>@kelcouch I'm sorry  at least it's Friday?</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-82cadee5-1b2f-43dc-97f7-7ea6b7ad3401')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-82cadee5-1b2f-43dc-97f7-7ea6b7ad3401 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-82cadee5-1b2f-43dc-97f7-7ea6b7ad3401');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-4ffb4a66-8566-464b-a89a-87c0322f981a\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-4ffb4a66-8566-464b-a89a-87c0322f981a')\"\n",
              "            title=\"Suggest charts.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-4ffb4a66-8566-464b-a89a-87c0322f981a button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "    </div>\n",
              "  </div>\n"
            ]
          },
          "metadata": {},
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Preprocessing the Data**"
      ],
      "metadata": {
        "id": "neUXuGaDT6oi"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import nltk\n",
        "nltk.download('stopwords')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "jUC_pm3KUKqh",
        "outputId": "616c9a8f-cc62-499d-af11-ac73782ad81d"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[nltk_data] Downloading package stopwords to /root/nltk_data...\n",
            "[nltk_data]   Unzipping corpora/stopwords.zip.\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "metadata": {},
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#Making all letters lowercase\n",
        "data['content'] = data['content'].apply(lambda x: \" \".join(x.lower() for x in x.split()))\n",
        "#Removing Punctuation, Symbols\n",
        "data['content'] = data['content'].str.replace('[^\\w\\s]',' ')\n",
        "#Removing Stop Words using NLTK\n",
        "from nltk.corpus import stopwords\n",
        "stop = stopwords.words('english')\n",
        "data['content'] = data['content'].apply(lambda x: \" \".join(x for x in x.split() if x not in stop))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "dRL0ZFAPT5qK",
        "outputId": "807bb8eb-8460-494d-e6b3-cef266b3d09c"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "<ipython-input-10-1c8527e3e95d>:4: FutureWarning: The default value of regex will change from True to False in a future version.\n",
            "  data['content'] = data['content'].str.replace('[^\\w\\s]',' ')\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "\n",
        "  import nltk\n",
        "  nltk.download('wordnet')"
      ],
      "metadata": {
        "id": "BlR-jr0fMfmW",
        "outputId": "fe10f842-3a8c-4445-aaec-e3dbf5b50571",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[nltk_data] Downloading package wordnet to /root/nltk_data...\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "metadata": {},
          "execution_count": 12
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#Lemmatisation\n",
        "from textblob import Word\n",
        "data['content'] = data['content'].apply(lambda x: \" \".join([Word(word).lemmatize() for word in x.split()]))\n",
        "#Correcting Letter Repetitions\n",
        "import re\n",
        "def de_repeat(text):\n",
        "    pattern = re.compile(r\"(.)\\1{2,}\")\n",
        "    return pattern.sub(r\"\\1\\1\", text)\n",
        "#%%\n",
        "data['content'] = data['content'].apply(lambda x: \" \".join(de_repeat(x) for x in x.split()))"
      ],
      "metadata": {
        "id": "PnOqDWkKUSgZ"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Next consideration is the idea that if a word is appearing only once in the entire sample of data, then it most likely has no influence in determining the sentiment of the text. Hence we can remove all the rarely occurring words from the dataset which are generally proper nouns and other insignificant words with respect to the current context."
      ],
      "metadata": {
        "id": "pBErUfXSNE_7"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Code to find the top 10,000 rarest words appearing in the data\n",
        "freq = pd.Series(' '.join(data['content']).split()).value_counts()[-10000:]\n",
        "# Removing all those rarely appearing words from the data\n",
        "freq = list(freq.index)\n",
        "data['content'] = data['content'].apply(lambda x: \" \".join(x for x in x.split() if x not in freq))"
      ],
      "metadata": {
        "id": "0OAHY7tSMsNu"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        " **Feature Extraction**"
      ],
      "metadata": {
        "id": "R6YfHAI1NUyp"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Once you make the text data clean, precise, and error-free, each tweet is represented by a group of keywords. Now, we need to perform ‘Feature Extraction’, i.e extracting some parameters from the data that can be presented numerically. In this article, we consider two different features, TF-IDF & Count Vectors (Remember, we need numeric data for the math!).\n",
        "Split the data into training and testing parts before performing feature extraction."
      ],
      "metadata": {
        "id": "31cAxSZvNdcn"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#Encoding output labels 'sadness' as '1' & 'happiness' as '0'\n",
        "from sklearn import preprocessing\n",
        "lbl_enc = preprocessing.LabelEncoder()\n",
        "y = lbl_enc.fit_transform(data.sentiment.values)\n",
        "# Splitting into training and testing data in 90:10 ratio\n",
        "from sklearn.model_selection import train_test_split\n",
        "X_train, X_val, y_train, y_val = train_test_split(data.content.values, y, stratify=y, random_state=42, test_size=0.1, shuffle=True)"
      ],
      "metadata": {
        "id": "FzhYC0zbNoBp"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Term Frequency-Inverse Document Frequency (TF-IDF):** This parameter gives the relative importance of a term in the data and is a measure of how frequently and rarely it appears in the text. This can be directly extracted in python as follows-"
      ],
      "metadata": {
        "id": "-EWegpPNNvG4"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Extracting TF-IDF parameters\n",
        "from sklearn.feature_extraction.text import TfidfVectorizer\n",
        "tfidf = TfidfVectorizer(max_features=1000, analyzer='word',ngram_range=(1,3))\n",
        "X_train_tfidf = tfidf.fit_transform(X_train)\n",
        "X_val_tfidf = tfidf.fit_transform(X_val)"
      ],
      "metadata": {
        "id": "N8zXqpS2Ny5v"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Count Vectors:** This is another feature we consider and as the name suggests we transform our tweet into an array having the count of appearances of each word in it. The intuition here is that the text that conveys similar emotions may have the same words repeated over and over again. This is more like the direct approach."
      ],
      "metadata": {
        "id": "VJqU-VAhN-DN"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Extracting Count Vectors Parameters\n",
        "from sklearn.feature_extraction.text import CountVectorizer\n",
        "count_vect = CountVectorizer(analyzer='word')\n",
        "count_vect.fit(data['content'])\n",
        "X_train_count =  count_vect.transform(X_train)\n",
        "X_val_count =  count_vect.transform(X_val)"
      ],
      "metadata": {
        "id": "taO5S4HkOHCp"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Training Our Models**"
      ],
      "metadata": {
        "id": "iaoeh0bNOQWN"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**First, let us build some models using the TF-IDF features-**"
      ],
      "metadata": {
        "id": "7-nF5fQ4OV_F"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**1-Multinomial Naive Bayes Classifier**"
      ],
      "metadata": {
        "id": "PSAdQhhkOpsm"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.metrics import accuracy_score\n",
        "# Model 1: Multinomial Naive Bayes Classifier\n",
        "from sklearn.naive_bayes import MultinomialNB\n",
        "nb = MultinomialNB()\n",
        "nb.fit(X_train_tfidf, y_train)\n",
        "y_pred = nb.predict(X_val_tfidf)\n",
        "print('naive bayes tfidf accuracy %s' % accuracy_score(y_pred, y_val))"
      ],
      "metadata": {
        "id": "hSt8Eez_OPpH",
        "outputId": "8f280315-0a10-4760-d17d-661792419062",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "naive bayes tfidf accuracy 0.48265895953757226\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Model 2: Linear SVM**"
      ],
      "metadata": {
        "id": "Lzw_OK7TOwk8"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.linear_model import SGDClassifier\n",
        "lsvm = SGDClassifier(alpha=0.001, random_state=5, max_iter=15, tol=None)\n",
        "lsvm.fit(X_train_tfidf, y_train)\n",
        "y_pred = lsvm.predict(X_val_tfidf)\n",
        "print('svm using tfidf accuracy %s' % accuracy_score(y_pred, y_val))\n"
      ],
      "metadata": {
        "id": "SGBJ0lmNO16w",
        "outputId": "1214f00b-fc42-4c36-d089-681c7506becd",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "svm using tfidf accuracy 0.5096339113680154\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Model 3: logistic regression**"
      ],
      "metadata": {
        "id": "bb-DMNL0PAVD"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Model 3: logistic regression\n",
        "from sklearn.linear_model import LogisticRegression\n",
        "logreg = LogisticRegression(C=1)\n",
        "logreg.fit(X_train_tfidf, y_train)\n",
        "y_pred = logreg.predict(X_val_tfidf)\n",
        "print('log reg tfidf accuracy %s' % accuracy_score(y_pred, y_val))"
      ],
      "metadata": {
        "id": "_xsZB9KiO-Gx",
        "outputId": "970ede64-74ca-43fd-f233-0ee8e2c5afdb",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "log reg tfidf accuracy 0.4951830443159923\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Model 4: Random Forest Classifier**"
      ],
      "metadata": {
        "id": "vdXEkYAoPGEW"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.ensemble import RandomForestClassifier\n",
        "rf = RandomForestClassifier(n_estimators=500)\n",
        "rf.fit(X_train_tfidf, y_train)\n",
        "y_pred = rf.predict(X_val_tfidf)\n",
        "print('random forest tfidf accuracy %s' % accuracy_score(y_pred, y_val))"
      ],
      "metadata": {
        "id": "TSGGe4wKPToR",
        "outputId": "c05249df-b306-4a2f-8c50-e0eaf734d05c",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "random forest tfidf accuracy 0.5057803468208093\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Now, let’s build models using count vectors features-**"
      ],
      "metadata": {
        "id": "76TUxkQYPbeQ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Model 1: Multinomial Naive Bayes Classifier\n",
        "from sklearn.naive_bayes import MultinomialNB\n",
        "nb = MultinomialNB()\n",
        "nb.fit(X_train_count, y_train)\n",
        "y_pred = nb.predict(X_val_count)\n",
        "print('naive bayes count vectors accuracy %s' % accuracy_score(y_pred, y_val))"
      ],
      "metadata": {
        "id": "Ta_E-pJ3Pg9I",
        "outputId": "2478bfc8-5415-4847-b720-ce227fe211b6",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "naive bayes count vectors accuracy 0.7745664739884393\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Model 2: Linear SVM\n",
        "from sklearn.linear_model import SGDClassifier\n",
        "lsvm = SGDClassifier(alpha=0.001, random_state=5, max_iter=15, tol=None)\n",
        "lsvm.fit(X_train_count, y_train)\n",
        "y_pred = lsvm.predict(X_val_count)\n",
        "print('lsvm using count vectors accuracy %s' % accuracy_score(y_pred, y_val))"
      ],
      "metadata": {
        "id": "x6gjdRvtPkv8",
        "outputId": "4561c5bd-f4f2-4a64-f2bc-7d8bc9b4adbe",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "lsvm using count vectors accuracy 0.7832369942196532\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Model 3: Logistic Regression\n",
        "from sklearn.linear_model import LogisticRegression\n",
        "logreg = LogisticRegression(C=1)\n",
        "logreg.fit(X_train_count, y_train)\n",
        "y_pred = logreg.predict(X_val_count)\n",
        "print('log reg count vectors accuracy %s' % accuracy_score(y_pred, y_val))"
      ],
      "metadata": {
        "id": "lGEX4YYZPp4L",
        "outputId": "630f26da-e050-4ecd-bf07-41574bf0b829",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "log reg count vectors accuracy 0.7832369942196532\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Model 4: Random Forest Classifier\n",
        "from sklearn.ensemble import RandomForestClassifier\n",
        "rf = RandomForestClassifier(n_estimators=500)\n",
        "rf.fit(X_train_count, y_train)\n",
        "y_pred = rf.predict(X_val_count)\n",
        "print('random forest with count vectors accuracy %s' % accuracy_score(y_pred, y_val))"
      ],
      "metadata": {
        "id": "uJ-tsXxIPthY",
        "outputId": "e485318b-6870-4cc3-fbba-8434af144825",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "random forest with count vectors accuracy 0.7466281310211946\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "By using count vectors, we have a significant improvement in performance. The best model, linear SVM achieved up to 79.28% accuracy.\n",
        "\n",
        "This might be because of the nature of this specific dataset where the emotion of the text is heavily dependent on the presence of some significant adjectives.\n",
        "\n",
        "Let us now test how it performs in reality by giving this model some random text input."
      ],
      "metadata": {
        "id": "btvam4_7P0Dx"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#Below are 8 random statements.\n",
        "#The first 4 depict happiness\n",
        "#The last 4 depict sadness\n",
        "tweets = pd.DataFrame(['I am very happy today! The atmosphere looks cheerful',\n",
        "'Things are looking great. It was such a good day',\n",
        "'Success is right around the corner. Lets celebrate this victory',\n",
        "'Everything is more beautiful when you experience them with a smile!',\n",
        "'Now this is my worst, okay? But I am gonna get better.',\n",
        "'I am tired, boss. Tired of being on the road, lonely as a sparrow in the rain. I am tired of all the pain I feel',\n",
        "'This is quite depressing. I am filled with sorrow',\n",
        "'His death broke my heart. It was a sad day'])\n",
        "# Doing some preprocessing on these tweets as done before\n",
        "tweets[0] = tweets[0].str.replace('[^\\w\\s]',' ')\n",
        "from nltk.corpus import stopwords\n",
        "stop = stopwords.words('english')\n",
        "tweets[0] = tweets[0].apply(lambda x: \" \".join(x for x in x.split() if x not in stop))\n",
        "from textblob import Word\n",
        "tweets[0] = tweets[0].apply(lambda x: \" \".join([Word(word).lemmatize() for word in x.split()]))\n",
        "# Extracting Count Vectors feature from our tweets\n",
        "tweet_count = count_vect.transform(tweets[0])\n",
        "#Predicting the emotion of the tweet using our already trained linear SVM\n",
        "tweet_pred = lsvm.predict(tweet_count)\n",
        "print(tweet_pred)\n"
      ],
      "metadata": {
        "id": "0fh4fGMfQClc",
        "outputId": "3beae657-7ad5-4189-b8b8-eb14472e49ac",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[0 0 0 0 1 1 1 1]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "<ipython-input-29-8f3f43198744>:13: FutureWarning: The default value of regex will change from True to False in a future version.\n",
            "  tweets[0] = tweets[0].str.replace('[^\\w\\s]',' ')\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **📚2-Machine Translation**"
      ],
      "metadata": {
        "id": "AZ_VY1HGYuS-"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## **Example 1**"
      ],
      "metadata": {
        "id": "yKzzToBnNRaJ"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Machine translation using k-nearest neighbors (KNN) in natural language processing (NLP) involves finding the nearest neighbors of a source sentence in a multilingual corpus and using their translations as the translation for the source sentence. Here's a simple example of how you can implement machine translation using KNN in Python:"
      ],
      "metadata": {
        "id": "DLBpcy7WMXjV"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.feature_extraction.text import CountVectorizer\n",
        "from sklearn.neighbors import NearestNeighbors\n",
        "import numpy as np\n",
        "\n",
        "# Sample multilingual corpus\n",
        "source_sentences = [\n",
        "    \"hello world\",\n",
        "    \"good morning\",\n",
        "    \"how are you\",\n",
        "    \"thank you\",\n",
        "    \"have a nice day\"\n",
        "]\n",
        "\n",
        "target_sentences = [\n",
        "    \"hola mundo\",\n",
        "    \"buenos días\",\n",
        "    \"cómo estás\",\n",
        "    \"gracias\",\n",
        "    \"que tengas un buen día\"\n",
        "]\n"
      ],
      "metadata": {
        "id": "g9kAoPhEMPrb"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Vectorize the source sentences\n",
        "vectorizer = CountVectorizer()\n",
        "X = vectorizer.fit_transform(source_sentences)\n",
        "X"
      ],
      "metadata": {
        "id": "X-krKiBkMk6s",
        "outputId": "5b7825b7-814a-4371-db0f-97519f5c3ad9",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<5x11 sparse matrix of type '<class 'numpy.int64'>'\n",
              "\twith 12 stored elements in Compressed Sparse Row format>"
            ]
          },
          "metadata": {},
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Fit KNN model\n",
        "knn = NearestNeighbors(n_neighbors=1, metric='cosine')\n",
        "knn.fit(X)"
      ],
      "metadata": {
        "id": "EIZj3r0kMvyC",
        "outputId": "3ba8180c-77f7-46a5-da38-228115f5b446",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 75
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "NearestNeighbors(metric='cosine', n_neighbors=1)"
            ],
            "text/html": [
              "<style>#sk-container-id-1 {color: black;background-color: white;}#sk-container-id-1 pre{padding: 0;}#sk-container-id-1 div.sk-toggleable {background-color: white;}#sk-container-id-1 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-container-id-1 label.sk-toggleable__label-arrow:before {content: \"▸\";float: left;margin-right: 0.25em;color: #696969;}#sk-container-id-1 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-container-id-1 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-container-id-1 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-container-id-1 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-container-id-1 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-container-id-1 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: \"▾\";}#sk-container-id-1 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-container-id-1 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-container-id-1 div.sk-estimator:hover {background-color: #d4ebff;}#sk-container-id-1 div.sk-parallel-item::after {content: \"\";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-container-id-1 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-container-id-1 div.sk-serial::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: 0;}#sk-container-id-1 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;position: relative;}#sk-container-id-1 div.sk-item {position: relative;z-index: 1;}#sk-container-id-1 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;position: relative;}#sk-container-id-1 div.sk-item::before, #sk-container-id-1 div.sk-parallel-item::before {content: \"\";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 0;bottom: 0;left: 50%;z-index: -1;}#sk-container-id-1 div.sk-parallel-item {display: flex;flex-direction: column;z-index: 1;position: relative;background-color: white;}#sk-container-id-1 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-container-id-1 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-container-id-1 div.sk-parallel-item:only-child::after {width: 0;}#sk-container-id-1 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;}#sk-container-id-1 div.sk-label label {font-family: monospace;font-weight: bold;display: inline-block;line-height: 1.2em;}#sk-container-id-1 div.sk-label-container {text-align: center;}#sk-container-id-1 div.sk-container {/* jupyter's `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-container-id-1 div.sk-text-repr-fallback {display: none;}</style><div id=\"sk-container-id-1\" class=\"sk-top-container\"><div class=\"sk-text-repr-fallback\"><pre>NearestNeighbors(metric=&#x27;cosine&#x27;, n_neighbors=1)</pre><b>In a Jupyter environment, please rerun this cell to show the HTML representation or trust the notebook. <br />On GitHub, the HTML representation is unable to render, please try loading this page with nbviewer.org.</b></div><div class=\"sk-container\" hidden><div class=\"sk-item\"><div class=\"sk-estimator sk-toggleable\"><input class=\"sk-toggleable__control sk-hidden--visually\" id=\"sk-estimator-id-1\" type=\"checkbox\" checked><label for=\"sk-estimator-id-1\" class=\"sk-toggleable__label sk-toggleable__label-arrow\">NearestNeighbors</label><div class=\"sk-toggleable__content\"><pre>NearestNeighbors(metric=&#x27;cosine&#x27;, n_neighbors=1)</pre></div></div></div></div></div>"
            ]
          },
          "metadata": {},
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Function to translate a given sentence\n",
        "def translate(sentence):\n",
        "    # Vectorize the input sentence\n",
        "    input_vector = vectorizer.transform([sentence])\n",
        "    # Find the nearest neighbor\n",
        "    nearest_neighbor_idx = knn.kneighbors(input_vector, return_distance=False)[0][0]\n",
        "    # Return the translation\n",
        "    return target_sentences[nearest_neighbor_idx]\n"
      ],
      "metadata": {
        "id": "JMl3r0bSM0oQ"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Example usage\n",
        "input_sentence = \"hello world\"\n",
        "translation = translate(input_sentence)\n",
        "print(\"Input:\", input_sentence)\n",
        "print(\"Translation:\", translation)"
      ],
      "metadata": {
        "id": "s97VYQe1M5-D",
        "outputId": "6a39336e-ec72-4bde-ccaa-b8c8acf2316d",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Input: hello world\n",
            "Translation: hola mundo\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## **Locality sensitive hashing**"
      ],
      "metadata": {
        "id": "jmYrk44At-cd"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Implementing machine translation using k-nearest neighbors (KNN) with Locality Sensitive Hashing (LSH) in Python can be a complex task. However, I can provide you with a basic outline of how you might approach it.\n",
        "\n",
        "First, you'll need a dataset of parallel texts, i.e., sentences or documents in one language paired with their translations in another language. You can use a dataset like Europarl or Tatoeba for this purpose.\n",
        "\n",
        "Here's a simplified outline of how you could implement this:"
      ],
      "metadata": {
        "id": "j-SZkcEHuW9F"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "1- Preprocessing:\n",
        "\n",
        "* Tokenize your text.\n",
        "\n",
        "* Convert words to vectors using techniques like TF-IDF or word embeddings.\n",
        "\n",
        "2- Locality Sensitive Hashing (LSH):\n",
        "\n",
        "Use a library like nearpy or datasketch for LSH implementation.\n",
        "Hash your vectors using LSH.\n",
        "K-Nearest Neighbors (KNN):\n",
        "\n",
        "Implement a KNN search using the hashed vectors.\n",
        "You can use libraries like scikit-learn for KNN.\n",
        "Translation:\n",
        "\n",
        "Once you have found the nearest neighbors for a given input sentence in the source language, retrieve the translations of these nearest neighbors in the target language.\n",
        "Below is a basic example of how you might implement this:"
      ],
      "metadata": {
        "id": "M8OCJlVmubaI"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install nearpy"
      ],
      "metadata": {
        "id": "lCYrMa-bvCwe",
        "outputId": "4ee4c213-e1be-4a7a-fa6d-eafd939893f2",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Collecting nearpy\n",
            "  Downloading NearPy-1.0.0-py2.py3-none-any.whl (64 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m64.4/64.4 kB\u001b[0m \u001b[31m1.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hCollecting bitarray (from nearpy)\n",
            "  Downloading bitarray-2.9.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (288 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m288.3/288.3 kB\u001b[0m \u001b[31m12.9 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: future in /usr/local/lib/python3.10/dist-packages (from nearpy) (0.18.3)\n",
            "Requirement already satisfied: numpy in /usr/local/lib/python3.10/dist-packages (from nearpy) (1.25.2)\n",
            "Requirement already satisfied: scipy in /usr/local/lib/python3.10/dist-packages (from nearpy) (1.11.4)\n",
            "Installing collected packages: bitarray, nearpy\n",
            "Successfully installed bitarray-2.9.2 nearpy-1.0.0\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.feature_extraction.text import TfidfVectorizer\n",
        "from sklearn.neighbors import NearestNeighbors\n",
        "from nearpy import Engine\n",
        "from nearpy.hashes import RandomBinaryProjections\n",
        "import numpy as np\n",
        "\n",
        "# Example parallel text data\n",
        "source_texts = ['This is an example sentence.', 'Another example sentence.']\n",
        "target_texts = ['Ceci est une phrase d exemple.', 'Une autre phrase d exemple.']\n",
        "\n",
        "# TF-IDF Vectorization\n",
        "tfidf_vectorizer = TfidfVectorizer()\n",
        "source_vectors = tfidf_vectorizer.fit_transform(source_texts)\n",
        "\n",
        "# Locality Sensitive Hashing (LSH)\n",
        "dim = source_vectors.shape[1]\n",
        "num_projections = 2  # Number of LSH projections\n",
        "rbp = RandomBinaryProjections('rbp', num_projections)\n",
        "engine = Engine(dim, lshashes=[rbp])\n",
        "\n",
        "# Add vectors to LSH index\n",
        "for i, vector in enumerate(source_vectors):\n",
        "    engine.store_vector(vector.toarray().ravel(), data=i)\n",
        "\n",
        "# KNN search\n",
        "knn = NearestNeighbors(n_neighbors=1, metric='euclidean')\n",
        "knn.fit(source_vectors)\n",
        "\n",
        "# Example input sentence\n",
        "input_sentence = 'Yet another example sentence.'\n",
        "\n",
        "# Transform input sentence to vector\n",
        "input_vector = tfidf_vectorizer.transform([input_sentence])\n",
        "\n",
        "# LSH search\n",
        "matches = list(engine.neighbours(input_vector.toarray().ravel()))\n",
        "\n",
        "# KNN search\n",
        "nearest_neighbor_index = knn.kneighbors(input_vector)[1][0][0]\n",
        "\n",
        "# Get translation\n",
        "translation = target_texts[nearest_neighbor_index]\n",
        "\n",
        "print(\"Input:\", input_sentence)\n",
        "print(\"Translation:\", translation)\n"
      ],
      "metadata": {
        "id": "psBYkzxFuzEn",
        "outputId": "25302bde-ec83-4f5a-f089-b5ea6837928a",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Input: Yet another example sentence.\n",
            "Translation: Une autre phrase d exemple.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **📚Autocorrect and Mininum Edit Distance[11]**"
      ],
      "metadata": {
        "id": "aHtmZu3R8NwZ"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "**Example 1**"
      ],
      "metadata": {
        "id": "FY-GvLYn8cM5"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def edit_distance(word1, word2):\n",
        "    m, n = len(word1), len(word2)\n",
        "    dp = [[0] * (n + 1) for _ in range(m + 1)]\n",
        "\n",
        "    for i in range(m + 1):\n",
        "        for j in range(n + 1):\n",
        "            if i == 0:\n",
        "                dp[i][j] = j\n",
        "            elif j == 0:\n",
        "                dp[i][j] = i\n",
        "            elif word1[i - 1] == word2[j - 1]:\n",
        "                dp[i][j] = dp[i - 1][j - 1]\n",
        "            else:\n",
        "                dp[i][j] = 1 + min(dp[i - 1][j],    # Deletion\n",
        "                                   dp[i][j - 1],    # Insertion\n",
        "                                   dp[i - 1][j - 1])  # Substitution\n",
        "    return dp[m][n]"
      ],
      "metadata": {
        "id": "teiYsiE_8a4r"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "def autocorrect(input_word, dictionary):\n",
        "    min_distance = float('inf')\n",
        "    corrected_word = input_word\n",
        "\n",
        "    for word in dictionary:\n",
        "        distance = edit_distance(input_word, word)\n",
        "        if distance < min_distance:\n",
        "            min_distance = distance\n",
        "            corrected_word = word\n",
        "\n",
        "    return corrected_word"
      ],
      "metadata": {
        "id": "4GEPYnnD8maM"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Example dictionary\n",
        "dictionary = [\"apple\", \"banana\", \"orange\", \"grape\", \"watermelon\", \"blueberry\"]\n"
      ],
      "metadata": {
        "id": "Z6RcYLM381_i"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Test the autocorrect function\n",
        "misspelled_word = \"appl\"\n",
        "corrected_word = autocorrect(misspelled_word, dictionary)\n",
        "print(f\"Did you mean: {corrected_word}?\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_KcnmyEu86h0",
        "outputId": "2cbd0de8-c043-4f81-d09d-3d344c8f3fff"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Did you mean: apple?\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **📚Chapter2: Part of Speech Tagging and Hidden Markov Models**"
      ],
      "metadata": {
        "id": "Ls9q-d6TTzez"
      }
    },
    {
      "cell_type": "code",
      "source": [
        " # Viterbi Algorithm for POS Tagging in Google Colab\n",
        "\n",
        "import numpy as np\n",
        "\n",
        "# 1. States (POS tags) and Probabilities\n",
        "states = ['Noun', 'Verb', 'Adjective']\n",
        "\n",
        "# Initial probabilities for each tag\n",
        "initial_probabilities = [0.5, 0.3, 0.2]  # Example probabilities\n",
        "\n",
        "# Transition probabilities: probabilities of moving from one state (tag) to another\n",
        "transition_probabilities = np.array([\n",
        "    [0.4, 0.3, 0.3],  # Transition probabilities from Noun\n",
        "    [0.2, 0.5, 0.3],  # Transition probabilities from Verb\n",
        "    [0.1, 0.2, 0.7]   # Transition probabilities from Adjective\n",
        "])\n",
        "\n",
        "# Emission probabilities: probabilities of a word being associated with each POS tag\n",
        "emission_probabilities = np.array([\n",
        "    [0.5, 0.1, 0.4],  # Emission probabilities for Noun\n",
        "    [0.2, 0.6, 0.2],  # Emission probabilities for Verb\n",
        "    [0.1, 0.2, 0.7]   # Emission probabilities for Adjective\n",
        "])\n",
        "\n",
        "# 2. Viterbi Algorithm Functions\n",
        "\n",
        "def initialize_viterbi(T):\n",
        "    \"\"\"Initializes the Viterbi matrices.\"\"\"\n",
        "    num_states = len(states)\n",
        "    C = np.zeros((num_states, T))  # Probability matrix\n",
        "    D = np.zeros((num_states, T), dtype=int)  # Backpointer matrix\n",
        "\n",
        "    for i in range(num_states):\n",
        "        C[i][0] = initial_probabilities[i] * emission_probabilities[i][0]\n",
        "        D[i][0] = 0\n",
        "\n",
        "    return C, D\n",
        "\n",
        "def forward_pass(C, D, T):\n",
        "    \"\"\"Populates the Viterbi matrices using the forward pass.\"\"\"\n",
        "    num_states = len(states)\n",
        "\n",
        "    for t in range(1, T):\n",
        "        for s in range(num_states):\n",
        "            max_prob = -1\n",
        "            best_state = 0\n",
        "            for prev_state in range(num_states):\n",
        "                prob = C[prev_state][t-1] * transition_probabilities[prev_state][s] * emission_probabilities[s][t % 3]\n",
        "                if prob > max_prob:\n",
        "                    max_prob = prob\n",
        "                    best_state = prev_state\n",
        "\n",
        "            C[s][t] = max_prob\n",
        "            D[s][t] = best_state\n",
        "\n",
        "    return C, D\n",
        "\n",
        "def backward_pass(C, D, T):\n",
        "    \"\"\"Traces back the best sequence of POS tags using the backpointer matrix.\"\"\"\n",
        "    best_path = np.zeros(T, dtype=int)\n",
        "    best_path[T-1] = np.argmax(C[:, T-1])\n",
        "\n",
        "    for t in range(T-2, -1, -1):\n",
        "        best_path[t] = D[best_path[t+1]][t+1]\n",
        "\n",
        "    best_tags = [states[state] for state in best_path]\n",
        "    return best_tags\n",
        "\n",
        "def viterbi_algorithm(sentence_length):\n",
        "    \"\"\"Runs the Viterbi algorithm on a sequence.\"\"\"\n",
        "    C, D = initialize_viterbi(sentence_length)\n",
        "    C, D = forward_pass(C, D, sentence_length)\n",
        "    best_tags = backward_pass(C, D, sentence_length)\n",
        "    return best_tags\n",
        "\n",
        "# 3. POS Tagging Function\n",
        "\n",
        "def tag_sentence(sentence):\n",
        "    \"\"\"Tags a sentence using the Viterbi algorithm.\"\"\"\n",
        "    words = sentence.split()\n",
        "    sentence_length = len(words)\n",
        "    best_tags = viterbi_algorithm(sentence_length)\n",
        "\n",
        "    print(\"\\nSentence:\", sentence)\n",
        "    print(\"Tags:\", best_tags)\n",
        "\n",
        "# 4. Example Sentences\n",
        "sentences = [\n",
        "    \"I run fast\",\n",
        "    \"The dog barks\",\n",
        "    \"He is tall\"\n",
        "]\n",
        "\n",
        "# 5. Tag Each Sentence\n",
        "for sentence in sentences:\n",
        "    tag_sentence(sentence)\n",
        "\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "iPcdhE0ZUC2_",
        "outputId": "2150b54a-4631-4e69-84d8-cd5b92f2cf23"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\n",
            "Sentence: I run fast\n",
            "Tags: ['Noun', 'Verb', 'Adjective']\n",
            "\n",
            "Sentence: The dog barks\n",
            "Tags: ['Noun', 'Verb', 'Adjective']\n",
            "\n",
            "Sentence: He is tall\n",
            "Tags: ['Noun', 'Verb', 'Adjective']\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **3- Named Entity Recognition (NER)**\n"
      ],
      "metadata": {
        "id": "9pGw_9s5hZHa"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import spacy\n",
        "\n",
        "# Load the pre-trained model\n",
        "nlp = spacy.load(\"en_core_web_sm\")\n",
        "\n",
        "# Define text to be analyzed\n",
        "text = \"Barack Obama visited the White House today\"\n",
        "\n",
        "# Process the text with the model\n",
        "doc = nlp(text)\n",
        "\n",
        "# Extract named entities\n",
        "for ent in doc.ents:\n",
        "    print(ent.text, ent.label_)"
      ],
      "metadata": {
        "id": "LoRmHPOSiht4",
        "outputId": "8074ae70-1583-42a2-a7ec-9187b8936bf7",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Barack Obama PERSON\n",
            "the White House ORG\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "#<p style=\"font-family:JetBrains Mono; font-weight:normal; letter-spacing: 1px; color:#207d06; font-size:100%; text-align:left;padding: 0px; border-bottom: 3px solid #207d06;\">**📚Chapter8 :Autocomplete and Language models**</p>"
      ],
      "metadata": {
        "id": "JThwYJcruYq3"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## **Creating and Using N-gram Language Models for Text Prediction and Generation**"
      ],
      "metadata": {
        "id": "gtvblCx5uf5i"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Step 1: Sample Corpus and Preprocessing\n",
        "We'll start with a small sample corpus to build the count and probability matrices."
      ],
      "metadata": {
        "id": "Uq2azH2VuxCy"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import numpy as np\n",
        "from collections import defaultdict, Counter\n",
        "\n",
        "# Sample corpus\n",
        "corpus = [\n",
        "    \"I study machine learning\",\n",
        "    \"I learn machine learning\",\n",
        "    \"machine learning is interesting\",\n",
        "    \"I study NLP\",\n",
        "    \"NLP is a branch of machine learning\"\n",
        "]\n",
        "\n",
        "# Preprocess the corpus\n",
        "processed_corpus = [sentence.lower().split() for sentence in corpus]\n"
      ],
      "metadata": {
        "id": "aJad2HwpuqoK"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Step 2: Create the Bigram Count Matrix\n",
        "We’ll count how often each bigram (two-word sequence) appears across the corpus."
      ],
      "metadata": {
        "id": "HQMoUf7tu7O-"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Initialize count dictionary\n",
        "bigram_counts = defaultdict(Counter)\n",
        "\n",
        "# Populate bigram count dictionary\n",
        "for sentence in processed_corpus:\n",
        "    for i in range(len(sentence) - 1):\n",
        "        first_word, second_word = sentence[i], sentence[i + 1]\n",
        "        bigram_counts[first_word][second_word] += 1\n",
        "\n",
        "# Display count matrix\n",
        "print(\"Bigram Count Matrix:\")\n",
        "for first_word, second_words in bigram_counts.items():\n",
        "    print(f\"{first_word}: {dict(second_words)}\")\n"
      ],
      "metadata": {
        "id": "gKnkX740u-kp",
        "outputId": "fdce528a-c66e-463a-b2d3-b6177ae3bd57",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Bigram Count Matrix:\n",
            "i: {'study': 2, 'learn': 1}\n",
            "study: {'machine': 1, 'nlp': 1}\n",
            "machine: {'learning': 4}\n",
            "learn: {'machine': 1}\n",
            "learning: {'is': 1}\n",
            "is: {'interesting': 1, 'a': 1}\n",
            "nlp: {'is': 1}\n",
            "a: {'branch': 1}\n",
            "branch: {'of': 1}\n",
            "of: {'machine': 1}\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Step 3: Calculate the Probability Matrix\n",
        "To create the probability matrix, we normalize the counts for each bigram by dividing by the total counts for each first word."
      ],
      "metadata": {
        "id": "Wo0R_DiHvFtr"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Convert counts to probabilities\n",
        "bigram_probabilities = defaultdict(dict)\n",
        "\n",
        "for first_word, second_words in bigram_counts.items():\n",
        "    total_count = sum(second_words.values())\n",
        "    for second_word, count in second_words.items():\n",
        "        bigram_probabilities[first_word][second_word] = count / total_count\n",
        "\n",
        "# Display probability matrix\n",
        "print(\"\\nBigram Probability Matrix:\")\n",
        "for first_word, second_words in bigram_probabilities.items():\n",
        "    print(f\"{first_word}: {second_words}\")\n"
      ],
      "metadata": {
        "id": "LASJaYl0vI98",
        "outputId": "885fcbb3-1d23-455a-fccc-740430699bb3",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\n",
            "Bigram Probability Matrix:\n",
            "i: {'study': 0.6666666666666666, 'learn': 0.3333333333333333}\n",
            "study: {'machine': 0.5, 'nlp': 0.5}\n",
            "machine: {'learning': 1.0}\n",
            "learn: {'machine': 1.0}\n",
            "learning: {'is': 1.0}\n",
            "is: {'interesting': 0.5, 'a': 0.5}\n",
            "nlp: {'is': 1.0}\n",
            "a: {'branch': 1.0}\n",
            "branch: {'of': 1.0}\n",
            "of: {'machine': 1.0}\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Step 4: Predict Next Word\n",
        "Using the probability matrix, we can predict the next word based on the previous word."
      ],
      "metadata": {
        "id": "H7-p-d61vN3H"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Function to predict next word\n",
        "def predict_next_word(current_word, bigram_probabilities):\n",
        "    if current_word in bigram_probabilities:\n",
        "        next_word = max(bigram_probabilities[current_word], key=bigram_probabilities[current_word].get)\n",
        "        return next_word\n",
        "    else:\n",
        "        return None\n",
        "\n",
        "# Predict the next word for a given word\n",
        "current_word = \"I\".lower()\n",
        "next_word = predict_next_word(current_word, bigram_probabilities)\n",
        "print(f\"\\nPredicted next word after '{current_word}': {next_word}\")\n"
      ],
      "metadata": {
        "id": "Q_q7GU8xvRa9",
        "outputId": "10acc21f-e86f-4c1a-96ef-9e89a48e579c",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\n",
            "Predicted next word after 'i': study\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Step 5: Generate Sentence\n",
        "We can use the bigram probabilities to generate a new sentence starting from a given word"
      ],
      "metadata": {
        "id": "sBhscRNYvW3F"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Function to generate a sentence\n",
        "def generate_sentence(start_word, bigram_probabilities, length=5):\n",
        "    sentence = [start_word]\n",
        "    current_word = start_word\n",
        "\n",
        "    for _ in range(length - 1):\n",
        "        next_word = predict_next_word(current_word, bigram_probabilities)\n",
        "        if next_word:\n",
        "            sentence.append(next_word)\n",
        "            current_word = next_word\n",
        "        else:\n",
        "            break  # Stop if there is no likely next word\n",
        "\n",
        "    return ' '.join(sentence)\n",
        "\n",
        "# Generate a sentence starting with 'I'\n",
        "start_word = \"I\".lower()\n",
        "generated_sentence = generate_sentence(start_word, bigram_probabilities)\n",
        "print(f\"\\nGenerated sentence: {generated_sentence}\")\n"
      ],
      "metadata": {
        "id": "zOZVc6qvvahu",
        "outputId": "74c5808a-fd6b-47f9-d4aa-6444bccd0e26",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\n",
            "Generated sentence: i study machine learning is\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **📚Chapter9: Word embedding with neural network**"
      ],
      "metadata": {
        "id": "RmOnm2X9tW6E"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import nltk\n",
        "nltk.download('punkt_tab')"
      ],
      "metadata": {
        "id": "8z7kmcr4ttZK",
        "outputId": "3ed25bee-daf3-4dea-fceb-95ad1ac978d9",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[nltk_data] Downloading package punkt_tab to /root/nltk_data...\n",
            "[nltk_data]   Unzipping tokenizers/punkt_tab.zip.\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "True"
            ]
          },
          "metadata": {},
          "execution_count": 2
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from gensim.models import Word2Vec\n",
        "from nltk.tokenize import sent_tokenize, word_tokenize\n",
        "# Sample text\n",
        "sample_text = \"The cat sat on the mat.\"\n",
        "data = [word_tokenize(sentence.lower()) for sentence in sent_tokenize(sample_text)]\n",
        "# Train Word2Vec model\n",
        "model = Word2Vec(data, vector_size=100, window=5, min_count=1, sg=0) # sg=0 for CBOW, sg=1 for Skip-Gram"
      ],
      "metadata": {
        "id": "5yzrgRL_thmH"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## **Continuous Bag-of-Words Model**"
      ],
      "metadata": {
        "id": "PKhx6BwfZFoQ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import numpy as np\n",
        "from collections import defaultdict"
      ],
      "metadata": {
        "id": "iyO4gNX3ZUOm"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Sample training data\n",
        "corpus = \"the quick brown fox jumps over the lazy dog\"\n",
        "words = corpus.split()\n",
        "vocab = set(words)\n",
        "vocab_size = len(vocab)\n",
        "word2idx = {word: idx for idx, word in enumerate(vocab)}\n",
        "idx2word = {idx: word for word, idx in word2idx.items()}"
      ],
      "metadata": {
        "id": "6Y7AoG4_Ze6V"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Hyperparameters\n",
        "window_size = 2\n",
        "embedding_size = 5\n",
        "learning_rate = 0.1\n",
        "epochs = 5000"
      ],
      "metadata": {
        "id": "Xa_1VYRqZj33"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Prepare training data\n",
        "def generate_training_data(words, window_size):\n",
        "    training_data = []\n",
        "    for i, word in enumerate(words):\n",
        "        context_words = []\n",
        "        for j in range(-window_size, window_size + 1):\n",
        "            if j != 0 and 0 <= i + j < len(words):\n",
        "                context_words.append(words[i + j])\n",
        "        training_data.append((context_words, word))\n",
        "    return training_data\n",
        "\n",
        "training_data = generate_training_data(words, window_size)"
      ],
      "metadata": {
        "id": "_QOgZWrHZozB"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "training_data"
      ],
      "metadata": {
        "id": "W7cIuJNaZs1v",
        "outputId": "6e140ec4-9870-44c0-ad02-6ea6664b764c",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[(['quick', 'brown'], 'the'),\n",
              " (['the', 'brown', 'fox'], 'quick'),\n",
              " (['the', 'quick', 'fox', 'jumps'], 'brown'),\n",
              " (['quick', 'brown', 'jumps', 'over'], 'fox'),\n",
              " (['brown', 'fox', 'over', 'the'], 'jumps'),\n",
              " (['fox', 'jumps', 'the', 'lazy'], 'over'),\n",
              " (['jumps', 'over', 'lazy', 'dog'], 'the'),\n",
              " (['over', 'the', 'dog'], 'lazy'),\n",
              " (['the', 'lazy'], 'dog')]"
            ]
          },
          "metadata": {},
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# One-hot encoding\n",
        "def one_hot_vector(word, vocab_size):\n",
        "    vector = np.zeros(vocab_size)\n",
        "    vector[word2idx[word]] = 1\n",
        "    return vector"
      ],
      "metadata": {
        "id": "vAGG_3VYZzRC"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Initialize weights\n",
        "W1 = np.random.rand(vocab_size, embedding_size)\n",
        "W2 = np.random.rand(embedding_size, vocab_size)"
      ],
      "metadata": {
        "id": "E0gzKnjfZ3QH"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Training CBOW model\n",
        "for epoch in range(epochs):\n",
        "    loss = 0\n",
        "    for context_words, target_word in training_data:\n",
        "        # Forward propagation\n",
        "        x = np.sum([one_hot_vector(w, vocab_size) for w in context_words], axis=0)  # Context vector\n",
        "        hidden_layer = np.dot(x, W1)  # Hidden layer\n",
        "        output_layer = np.dot(hidden_layer, W2)  # Output layer\n",
        "        y_pred = np.exp(output_layer) / np.sum(np.exp(output_layer))  # Softmax activation\n",
        "\n",
        "        # Compute loss (Cross-entropy)\n",
        "        y_true = one_hot_vector(target_word, vocab_size)\n",
        "        loss += -np.sum(y_true * np.log(y_pred))\n",
        "\n",
        "        # Backpropagation\n",
        "        error = y_pred - y_true\n",
        "        dW2 = np.outer(hidden_layer, error)\n",
        "        dW1 = np.outer(x, np.dot(W2, error))\n",
        "\n",
        "        # Update weights\n",
        "        W1 -= learning_rate * dW1\n",
        "        W2 -= learning_rate * dW2\n",
        "\n",
        "    if epoch % 1000 == 0:\n",
        "        print(f\"Epoch {epoch}, Loss: {loss:.4f}\")"
      ],
      "metadata": {
        "id": "K6tBNOLCZ-Uf",
        "outputId": "4f4cd39e-8bc6-4014-ca69-751c6e222e45",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 0, Loss: 25.9321\n",
            "Epoch 1000, Loss: 0.0059\n",
            "Epoch 2000, Loss: 0.0026\n",
            "Epoch 3000, Loss: 0.0016\n",
            "Epoch 4000, Loss: 0.0012\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "\n",
        "# Word embeddings\n",
        "print(\"Word Embeddings:\")\n",
        "for word in vocab:\n",
        "    print(f\"{word}: {W1[word2idx[word]]}\")"
      ],
      "metadata": {
        "id": "f-euBT36aFFH",
        "outputId": "24abd26b-79c4-439f-bffd-6fd9c9cc8ef4",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Word Embeddings:\n",
            "over: [-1.11307266 -0.84451954  0.51553881  3.47643947 -2.23707504]\n",
            "the: [ 0.52195244 -1.80162292 -0.68673763 -1.65068871  0.06566097]\n",
            "lazy: [-1.29979745  1.31832352 -1.59612135 -1.79088065  3.16849635]\n",
            "dog: [-1.60013546 -1.02817525 -0.86527102  0.55556261  1.63416753]\n",
            "jumps: [ 0.08481608  2.62373233  1.96392963  0.38098413 -0.32104825]\n",
            "fox: [ 2.25561309  0.17289425  1.98303075 -0.49024714 -0.7268657 ]\n",
            "quick: [-1.0153137   0.32171428  1.11143579  2.12311206  2.02558018]\n",
            "brown: [ 0.78071177  0.13448479 -1.2327252   0.57648234  0.46345289]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## **Extracting Word Embedding Vectors**"
      ],
      "metadata": {
        "id": "iW09qTp7OXM-"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import numpy as np\n",
        "\n",
        "# Example: assume these are your trained weights\n",
        "# N = embedding size, V = vocabulary size\n",
        "N = 5   # embedding dimension\n",
        "V = 10  # vocabulary size\n",
        "\n",
        "# Simulate trained weights for illustration\n",
        "np.random.seed(42)  # for reproducibility\n",
        "W1 = np.random.randn(N, V)  # shape: (N, V)\n",
        "W2 = np.random.randn(V, N)  # shape: (V, N)\n",
        "\n",
        "# Vocabulary list (example order)\n",
        "vocab = [\"am\", \"because\", \"happy\", \"I\", \"learning\", \"I'm\", \"coding\", \"AI\", \"chatbot\", \"learn\"]\n",
        "\n",
        "# Check dimensions\n",
        "assert W1.shape == (N, V)\n",
        "assert W2.shape == (V, N)\n",
        "assert len(vocab) == V\n",
        "\n",
        "# 1. Extract embeddings from input matrix W1\n",
        "# Column i of W1 = embedding for vocab[i]\n",
        "embeddings_W1 = W1.T  # shape: (V, N)\n",
        "\n",
        "# 2. Extract embeddings from output matrix W2\n",
        "# Row i of W2 = embedding for vocab[i]\n",
        "embeddings_W2 = W2    # shape: (V, N)\n",
        "\n",
        "# 3. Average W1.T and W2 to get final embeddings\n",
        "embeddings_avg = (embeddings_W1 + embeddings_W2) / 2  # shape: (V, N)\n",
        "\n",
        "# Store in a dictionary for easy lookup\n",
        "word_embeddings = {\n",
        "    \"W1_only\": {word: embeddings_W1[i] for i, word in enumerate(vocab)},\n",
        "    \"W2_only\": {word: embeddings_W2[i] for i, word in enumerate(vocab)},\n",
        "    \"averaged\": {word: embeddings_avg[i] for i, word in enumerate(vocab)}\n",
        "}\n",
        "\n",
        "# 🔍 Example: Display the averaged embedding for the word \"happy\"\n",
        "print(\"Averaged embedding for 'happy':\")\n",
        "print(word_embeddings[\"averaged\"][\"happy\"])\n"
      ],
      "metadata": {
        "id": "x4EtICd6Os6F",
        "outputId": "5eed188a-11a0-4038-e073-941fb23432f8",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Averaged embedding for 'happy':\n",
            "[ 0.08425715  0.02815165 -0.51940338 -0.60485192  0.34843877]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **4-Text Summarization**\n"
      ],
      "metadata": {
        "id": "zVCziQT5i0iw"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **5- Information Extraction**"
      ],
      "metadata": {
        "id": "Uydq3Wy2jXsL"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **6-Text Generation**\n"
      ],
      "metadata": {
        "id": "-yGrueEojiZP"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install -U sentence-transformers\n"
      ],
      "metadata": {
        "id": "hirPFYZKkhwe"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from transformers import GPT2Tokenizer, GPT2LMHeadModel\n",
        "import numpy as np\n",
        "\n",
        "\n",
        "# Load the GPT-2 model and tokenizer\n",
        "tokenizer = GPT2Tokenizer.from_pretrained(\"gpt2\")\n",
        "model = GPT2LMHeadModel.from_pretrained(\"gpt2\")\n",
        "\n",
        "# Define the prompt and generate text\n",
        "prompt = \"Once upon a time in a land far, far away\"\n",
        "generated_text = model.generate(input_ids=tokenizer.encode(prompt))\n",
        "\n",
        "# Decode the generated text\n",
        "generated_text = tokenizer.decode(generated_text)\n",
        "print(generated_text)"
      ],
      "metadata": {
        "id": "_2Co_QxFkPZI",
        "outputId": "3497d32a-9235-4611-bef4-5bdb271c8136",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 453
        }
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.\n",
            "Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.\n"
          ]
        },
        {
          "output_type": "error",
          "ename": "AttributeError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mAttributeError\u001b[0m                            Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-8-8eef2e77769a>\u001b[0m in \u001b[0;36m<cell line: 11>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      9\u001b[0m \u001b[0;31m# Define the prompt and generate text\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     10\u001b[0m \u001b[0mprompt\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m\"Once upon a time in a land far, far away\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 11\u001b[0;31m \u001b[0mgenerated_text\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgenerate\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0minput_ids\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mtokenizer\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mencode\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mprompt\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     12\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     13\u001b[0m \u001b[0;31m# Decode the generated text\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py\u001b[0m in \u001b[0;36mdecorate_context\u001b[0;34m(*args, **kwargs)\u001b[0m\n\u001b[1;32m    113\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0mdecorate_context\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    114\u001b[0m         \u001b[0;32mwith\u001b[0m \u001b[0mctx_factory\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 115\u001b[0;31m             \u001b[0;32mreturn\u001b[0m \u001b[0mfunc\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    116\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    117\u001b[0m     \u001b[0;32mreturn\u001b[0m \u001b[0mdecorate_context\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.10/dist-packages/transformers/generation/utils.py\u001b[0m in \u001b[0;36mgenerate\u001b[0;34m(self, inputs, generation_config, logits_processor, stopping_criteria, prefix_allowed_tokens_fn, synced_gpus, assistant_model, streamer, **kwargs)\u001b[0m\n\u001b[1;32m   1306\u001b[0m             \u001b[0minputs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mgeneration_config\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mbos_token_id\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmodel_kwargs\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1307\u001b[0m         )\n\u001b[0;32m-> 1308\u001b[0;31m         \u001b[0mbatch_size\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0minputs_tensor\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mshape\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1309\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1310\u001b[0m         \u001b[0;31m# 4. Define other model kwargs\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mAttributeError\u001b[0m: 'list' object has no attribute 'shape'"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **7-Text Clustering**\n"
      ],
      "metadata": {
        "id": "BSCQX2xxjq2I"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **8- Speech Recognition**\n"
      ],
      "metadata": {
        "id": "JotH9Cbajyyg"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **9- Text-to-Speech (TTS)**\n"
      ],
      "metadata": {
        "id": "fk7QY7Ujj9qI"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **References**"
      ],
      "metadata": {
        "id": "9GBsODuyD82C"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "[1-nlp-tutorial](https://github.com/bonzanini/nlp-tutorial/blob/master/notebooks/00%20Environment%20Test.ipynb)\n",
        "\n",
        "[2-Named Entity Recognition (NER)](https://thecleverprogrammer.com/2020/08/04/named-entity-recognition-ner/)\n",
        "\n",
        "[3-Modern Methods for Text Classification](https://github.com/NirantK/NLP_Quickbook/blob/master/05_Text_Classification.ipynb)\n",
        "\n",
        "[4-NLP_Quickbook](https://github.com/NirantK/NLP_Quickbook/tree/master)\n",
        "\n",
        "[5-The Super Duper NLP Repo](https://notebooks.quantumstat.com/)\n",
        "\n",
        "[6-01_OnePipeline_ManyClassifiers.ipynb](https://colab.research.google.com/github/practical-nlp/practical-nlp/blob/master/Ch4/01_OnePipeline_ManyClassifiers.ipynb#scrollTo=BYW_S3585xXF)\n",
        "\n",
        "[7-Two minutes NLP — 21 Learning Resources for Text Classification](https://medium.com/nlplanet/two-minutes-nlp-21-learning-resources-for-text-classification-b6f9c43793e1)\n",
        "\n",
        "[8-Building a Text Classifier App with Hugging Face, BERT, and Comet](https://heartbeat.comet.ml/building-a-text-classifier-app-with-hugging-face-bert-and-comet-278e4cd0d0aa)\n",
        "\n",
        "[9-Spam-Ham Email Detection](https://medium.com/@dssc2022yt/spam-ham-email-detection-d548a6d5869e)\n",
        "\n",
        "[10-Applied Machine Learning: Part 3](https://medium.com/the-research-nest/applied-machine-learning-part-3-3fd405842a18)\n",
        "\n",
        "[11-Autocorrect Feature Using NLP with Python](https://github.com/gvdnikhil/Autocorrect-Feature-using-NLP-with-Python)"
      ],
      "metadata": {
        "id": "KYNVMVp-FZDf"
      }
    }
  ]
}