notebooks/2.0_gradio_parallel_summaries.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# BUILDING NLP WEB APPS WITH GRADIO AND HUGGING FACE TRANSFORMERS\n",
    "\n",
    "What's interesting about the latest version of Gradio is that you can load multiple transformer models in one web app, either in parallel or in series, ie, compare different translation models in one app, or combine translation and summary models in one app.\n",
    "\n",
    "This is a really useful feature, given the rapidly growing number of transformer models available within as well as across different tasks. Within summarization, for instance, there are [at least 248 models](https://huggingface.co/models?pipeline_tag=summarization) on Hugging Face's model hub. How do you know which one works better for your use case? Well, one way is to directly compare how one performs against another when given the same text to summarize.\n",
    "\n",
    "Let's try that in this notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import gradio as gr\n",
    "import re\n",
    "import warnings\n",
    "\n",
    "\n",
    "from gradio.mix import Parallel\n",
    "from nltk.tokenize import sent_tokenize\n",
    "from transformers import (\n",
    "    AutoTokenizer,\n",
    "    AutoModel,\n",
    "    AutoModelForSeq2SeqLM,\n",
    "    Wav2Vec2ForCTC,\n",
    "    Wav2Vec2Tokenizer,\n",
    "    pipeline,\n",
    ")\n",
    "\n",
    "warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 1. DEFINE TEXT CLEANING AND SUMMARIZATION FUNCTIONS "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Tweak the text cleaning function further if you wish\n",
    "\n",
    "def clean_text(text):\n",
    "    text = text.encode(\"ascii\", errors=\"ignore\").decode(\n",
    "        \"ascii\"\n",
    "    )  # remove non-ascii, Chinese characters\n",
    "    text = re.sub(r\"\\n\", \" \", text)\n",
    "    text = re.sub(r\"\\n\\n\", \" \", text)\n",
    "    text = re.sub(r\"\\t\", \" \", text)\n",
    "    text = text.strip(\" \")\n",
    "    text = re.sub(\n",
    "        \" +\", \" \", text\n",
    "    ).strip()  # get rid of multiple spaces and replace with a single\n",
    "    return text   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.1 SUMMARIZATION VIA HUGGING FACE PIPELINE\n",
    "\n",
    "Hugging Face's [pipeline allows you to load up several summarization models](https://huggingface.co/transformers/main_classes/pipelines.html#transformers.SummarizationPipeline), from FB's Bart to Google's T5. For this example, I'll use the \"facebook/bart-large-cnn\" model, which has given decent results in my previous tests."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "pipeline_summ = pipeline(\n",
    "    \"summarization\",\n",
    "    model=\"facebook/bart-large-cnn\", # switch out to \"t5-small\" etc if you wish\n",
    "    tokenizer=\"facebook/bart-large-cnn\", # as above\n",
    "    framework=\"pt\",\n",
    ")\n",
    "\n",
    "# First of 2 summarization function\n",
    "def fb_summarizer(text):\n",
    "    input_text = clean_text(text)\n",
    "    results = pipeline_summ(input_text)\n",
    "    return results[0][\"summary_text\"]\n",
    "\n",
    "# First of 2 Gradio apps that we'll put in \"parallel\"\n",
    "summary1 = gr.Interface(\n",
    "    fn=fb_summarizer,\n",
    "    inputs=gr.inputs.Textbox(),\n",
    "    outputs=gr.outputs.Textbox(label=\"Summary by FB/Bart-large\"),\n",
    ")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.2 SUMMARIZATION VIA HUGGING FACE'S MODEL HUB\n",
    "\n",
    "Not all transformer models finetuned for summarization are compatible with the pipeline. The relatively [newer Pegasus models by Google](https://ai.googleblog.com/2020/06/pegasus-state-of-art-model-for.html) have to be loaded up the \"old-fashioned\" way as below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_name = \"google/pegasus-cnn_dailymail\" # Pegasus has a few variations; switch out as required\n",
    "\n",
    "# Second of 2 summarization function\n",
    "def google_summarizer(text):\n",
    "    input_text = clean_text(text)\n",
    "    \n",
    "    tokenizer_pegasus = AutoTokenizer.from_pretrained(model_name)\n",
    "\n",
    "    model_pegasus = AutoModelForSeq2SeqLM.from_pretrained(model_name)\n",
    "\n",
    "    batch = tokenizer_pegasus.prepare_seq2seq_batch(\n",
    "        input_text, truncation=True, padding=\"longest\", return_tensors=\"pt\"\n",
    "    )\n",
    "    translated = model_pegasus.generate(**batch)\n",
    "\n",
    "    pegasus_summary = tokenizer_pegasus.batch_decode(\n",
    "        translated, skip_special_tokens=True\n",
    "    )\n",
    "\n",
    "    return pegasus_summary[0]\n",
    "\n",
    "# Second of 2 Gradio apps that we'll put in \"parallel\"\n",
    "summary2 = gr.Interface(\n",
    "    fn=google_summarizer,\n",
    "    inputs=gr.inputs.Textbox(),\n",
    "    outputs=gr.outputs.Textbox(label=\"Summary by Google/Pegasus-CNN-Dailymail\"),\n",
    ")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2 LAUNCH SUMMARIZATION MODELS IN PARALLEL"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running locally at: http://127.0.0.1:7860/\n",
      "To create a public link, set `share=True` in `launch()`.\n",
      "Interface loading below...\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "        <iframe\n",
       "            width=\"900\"\n",
       "            height=\"500\"\n",
       "            src=\"http://127.0.0.1:7860/\"\n",
       "            frameborder=\"0\"\n",
       "            allowfullscreen\n",
       "        ></iframe>\n",
       "        "
      ],
      "text/plain": [
       "<IPython.lib.display.IFrame at 0x7fd2a09ceda0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "(<Flask 'gradio.networking'>, 'http://127.0.0.1:7860/', None)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Parallel(\n",
    "    summary1,\n",
    "    summary2,\n",
    "    title=\"Compare 2 AI Summarizers\",\n",
    "    inputs=gr.inputs.Textbox(lines=20, label=\"Paste some English text here\"),\n",
    ").launch()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}