notebooks/02_dense_retriever.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "18269eb9-193d-491b-9a4b-13e864146fb2",
   "metadata": {
    "tags": []
   },
   "source": [
    "# Embedding Based Retrieval: Text"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9392a9e3-8386-4402-9fdd-20ec81222bb7",
   "metadata": {
    "tags": []
   },
   "source": [
    "# About"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5a2733a6-577e-4997-8c2b-1c91b4756828",
   "metadata": {},
   "source": [
    "In this notebook we will do follows:\n",
    "- Create a text corpus by using captions\n",
    "- Use existing embedding model `sentence-transformers/all-MiniLM-L6-v2`\n",
    "- Explore the retrieval results of the model "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c4066b9a-d409-4529-910e-57812c57880b",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "239de404-8843-4a37-a7ba-cc02763ec71a",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "50850fda-e20a-407a-bf19-c2b2eb0bb816",
   "metadata": {},
   "source": [
    "# About the Embedding model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ff7964ce-ee3c-4cdf-b8f2-37dcf45006ab",
   "metadata": {},
   "source": [
    "About the model `sentence-transformers/all-MiniLM-L6-v2`\n",
    "\n",
    "\n",
    "From the [Docs](https://huggingface.co/sentence-transformers/all-MiniLM-L6-v2)\n",
    "\n",
    "It maps sentences & paragraphs to a `384` dimensional dense vector space and can be used for tasks like clustering or semantic search.\n",
    "\n",
    "The project aims to train sentence embedding models on very large sentence level datasets using a self-supervised contrastive learning objective.    \n",
    "\n",
    "We fine-tuned in on a 1B sentence pairs dataset. \n",
    "\n",
    "We use a contrastive learning objective: given a sentence from the pair, the model should predict which out of a set of randomly sampled other sentences, was actually paired with it in our dataset.\n",
    "\n",
    "\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "76cdf992-ddbc-41a4-87b0-2669bb577793",
   "metadata": {},
   "source": [
    "| **Dataset**                                                                                                                      | **Number of training tuples** |\n",
    "| -------------------------------------------------------------------------------------------------------------------------------- | ----------------------------- |\n",
    "| [Reddit comments (2015-2018)](https://github.com/PolyAI-LDN/conversational-datasets/tree/master/reddit)                          | 726,484,430                   |\n",
    "| [S2ORC](https://github.com/allenai/s2orc) Citation pairs (Abstracts)                                                             | 116,288,806                   |\n",
    "| [WikiAnswers](https://github.com/afader/oqa#wikianswers-corpus) Duplicate question pairs                                         | 77,427,422                    |\n",
    "| [PAQ](https://github.com/facebookresearch/PAQ) (Question, Answer) pairs                                                          | 64,371,441                    |\n",
    "| [S2ORC](https://github.com/allenai/s2orc) Citation pairs (Titles)                                                                | 52,603,982                    |\n",
    "| [S2ORC](https://github.com/allenai/s2orc) (Title, Abstract)                                                                      | 41,769,185                    |\n",
    "| [Stack Exchange](https://huggingface.co/datasets/flax-sentence-embeddings/stackexchange_xml) (Title, Body) pairs                 | 25,316,456                    |\n",
    "| [Stack Exchange](https://huggingface.co/datasets/flax-sentence-embeddings/stackexchange_xml) (Title+Body, Answer) pairs          | 21,396,559                    |\n",
    "| [Stack Exchange](https://huggingface.co/datasets/flax-sentence-embeddings/stackexchange_xml) (Title, Answer) pairs               | 21,396,559                    |\n",
    "| [MS MARCO](https://microsoft.github.io/msmarco/) triplets                                                                        | 9,144,553                     |\n",
    "| [GOOAQ: Open Question Answering with Diverse Answer Types](https://github.com/allenai/gooaq)                                     | 3,012,496                     |\n",
    "| [Yahoo Answers](https://www.kaggle.com/soumikrakshit/yahoo-answers-dataset) (Title, Answer)                                      | 1,198,260                     |\n",
    "| [Code Search](https://huggingface.co/datasets/code_search_net)                                                                   | 1,151,414                     |\n",
    "| [COCO](https://cocodataset.org/#home) Image captions                                                                             | 828,395                       |\n",
    "| [SPECTER](https://github.com/allenai/specter) citation triplets                                                                  | 684,100                       |\n",
    "| [Yahoo Answers](https://www.kaggle.com/soumikrakshit/yahoo-answers-dataset) (Question, Answer)                                   | 681,164                       |\n",
    "| [Yahoo Answers](https://www.kaggle.com/soumikrakshit/yahoo-answers-dataset) (Title, Question)                                    | 659,896                       |\n",
    "| [SearchQA](https://huggingface.co/datasets/search_qa)                                                                            | 582,261                       |\n",
    "| [Eli5](https://huggingface.co/datasets/eli5)                                                                                     | 325,475                       |\n",
    "| [Flickr 30k](https://shannon.cs.illinois.edu/DenotationGraph/)                                                                   | 317,695                       |\n",
    "| [Stack Exchange](https://huggingface.co/datasets/flax-sentence-embeddings/stackexchange_xml) Duplicate questions (titles)        | 304,525                       |\n",
    "| AllNLI ([SNLI](https://nlp.stanford.edu/projects/snli/) and [MultiNLI](https://cims.nyu.edu/~sbowman/multinli/)                  | 277,230                       |\n",
    "| [Stack Exchange](https://huggingface.co/datasets/flax-sentence-embeddings/stackexchange_xml) Duplicate questions (bodies)        | 250,519                       |\n",
    "| [Stack Exchange](https://huggingface.co/datasets/flax-sentence-embeddings/stackexchange_xml) Duplicate questions (titles+bodies) | 250,460                       |\n",
    "| [Sentence Compression](https://github.com/google-research-datasets/sentence-compression)                                         | 180,000                       |\n",
    "| [Wikihow](https://github.com/pvl/wikihow_pairs_dataset)                                                                          | 128,542                       |\n",
    "| [Altlex](https://github.com/chridey/altlex/)                                                                                     | 112,696                       |\n",
    "| [Quora Question Triplets](https://quoradata.quora.com/First-Quora-Dataset-Release-Question-Pairs)                                | 103,663                       |\n",
    "| [Simple Wikipedia](https://cs.pomona.edu/~dkauchak/simplification/)                                                              | 102,225                       |\n",
    "| [Natural Questions (NQ)](https://ai.google.com/research/NaturalQuestions)                                                        | 100,231                       |\n",
    "| [SQuAD2.0](https://rajpurkar.github.io/SQuAD-explorer/)                                                                          | 87,599                        |\n",
    "| [TriviaQA](https://huggingface.co/datasets/trivia_qa)                                                                            | 73,346                        |\n",
    "| **Total**                                                                                                                        | **1,170,060,424**             |"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "71cc82c8-43e0-4177-b850-543458b966ff",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0de4de3f-64c7-4f0e-b7b6-2a01ff4c6b94",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1e0ac3bd-0d3a-4567-94fc-01b10ab900b3",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "a615ea4c-3760-4b72-af7f-a84022f8c9c6",
   "metadata": {},
   "source": [
    "Other models listed in this page are also great candidates\n",
    "\n",
    "https://www.sbert.net/docs/pretrained_models.html#model-overview"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "63604938-8d33-44b8-9081-85c5db3a3cc8",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "1278e099-1a17-483e-a694-6ee7b4ba10d7",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "b352bcd9-d1c4-4846-b3d8-efd63ae3255d",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-26T12:48:58.231108Z",
     "iopub.status.busy": "2023-04-26T12:48:58.230264Z",
     "iopub.status.idle": "2023-04-26T12:49:08.291074Z",
     "shell.execute_reply": "2023-04-26T12:49:08.290201Z",
     "shell.execute_reply.started": "2023-04-26T12:48:58.231064Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sentence_transformers import SentenceTransformer\n",
    "\n",
    "import datasets\n",
    "import rich\n",
    "from IPython.display import Image, JSON\n",
    "from IPython.core.display import HTML\n",
    "import numpy as np\n",
    "\n",
    "from transformers import AutoTokenizer\n",
    "import ipywidgets as widgets\n",
    "from ipywidgets import interact\n",
    "import ipyplot\n",
    "import os"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a51e4d05-d325-4c8f-b184-d2aa634b7fcf",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "ff2d7f69-9a4b-41a7-8b38-0519f74fb1b8",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-26T12:49:08.293557Z",
     "iopub.status.busy": "2023-04-26T12:49:08.292892Z",
     "iopub.status.idle": "2023-04-26T12:49:08.616931Z",
     "shell.execute_reply": "2023-04-26T12:49:08.616139Z",
     "shell.execute_reply.started": "2023-04-26T12:49:08.293521Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "dset = datasets.load_from_disk(\"../data/processed\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "9856a687-dc58-4938-ba22-738bed860fdd",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-26T12:49:08.618370Z",
     "iopub.status.busy": "2023-04-26T12:49:08.617986Z",
     "iopub.status.idle": "2023-04-26T12:49:08.625324Z",
     "shell.execute_reply": "2023-04-26T12:49:08.624410Z",
     "shell.execute_reply.started": "2023-04-26T12:49:08.618336Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Dataset({\n",
       "    features: ['photo_id', 'photo_url', 'photo_image_url', 'photo_submitted_at', 'photo_featured', 'photo_width', 'photo_height', 'photo_aspect_ratio', 'photo_description', 'photographer_username', 'photographer_first_name', 'photographer_last_name', 'exif_camera_make', 'exif_camera_model', 'exif_iso', 'exif_aperture_value', 'exif_focal_length', 'exif_exposure_time', 'photo_location_name', 'photo_location_latitude', 'photo_location_longitude', 'photo_location_country', 'photo_location_city', 'stats_views', 'stats_downloads', 'ai_description', 'ai_primary_landmark_name', 'ai_primary_landmark_latitude', 'ai_primary_landmark_longitude', 'ai_primary_landmark_confidence', 'blur_hash', 'description_final', 'image', 'image_path_local'],\n",
       "    num_rows: 24992\n",
       "})"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "3e74356f-e0cb-4e80-b876-f58cd8940c88",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-26T12:49:08.627807Z",
     "iopub.status.busy": "2023-04-26T12:49:08.627482Z",
     "iopub.status.idle": "2023-04-26T12:49:08.964833Z",
     "shell.execute_reply": "2023-04-26T12:49:08.964142Z",
     "shell.execute_reply.started": "2023-04-26T12:49:08.627777Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\">{</span>\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'photo_id'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'XMyPniM9LF0'</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'photo_url'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'https://unsplash.com/photos/XMyPniM9LF0'</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'photo_image_url'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'https://images.unsplash.com/uploads/14119492946973137ce46/f1f2ebf3'</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'photo_submitted_at'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'2014-09-29 00:08:38.594364'</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'photo_featured'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'t'</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'photo_width'</span>: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">4272</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'photo_height'</span>: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">2848</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'photo_aspect_ratio'</span>: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1.5</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'photo_description'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'Woman exploring a forest'</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'photographer_username'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'michellespencer77'</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'photographer_first_name'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'Michelle'</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'photographer_last_name'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'Spencer'</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'exif_camera_make'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'Canon'</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'exif_camera_model'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'Canon EOS REBEL T3'</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'exif_iso'</span>: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">400.0</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'exif_aperture_value'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'1.8'</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'exif_focal_length'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'50.0'</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'exif_exposure_time'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'1/100'</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'photo_location_name'</span>: <span style=\"color: #800080; text-decoration-color: #800080; font-style: italic\">None</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'photo_location_latitude'</span>: <span style=\"color: #800080; text-decoration-color: #800080; font-style: italic\">None</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'photo_location_longitude'</span>: <span style=\"color: #800080; text-decoration-color: #800080; font-style: italic\">None</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'photo_location_country'</span>: <span style=\"color: #800080; text-decoration-color: #800080; font-style: italic\">None</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'photo_location_city'</span>: <span style=\"color: #800080; text-decoration-color: #800080; font-style: italic\">None</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'stats_views'</span>: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">2375421</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'stats_downloads'</span>: <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">6967</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'ai_description'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'woman walking in the middle of forest'</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'ai_primary_landmark_name'</span>: <span style=\"color: #800080; text-decoration-color: #800080; font-style: italic\">None</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'ai_primary_landmark_latitude'</span>: <span style=\"color: #800080; text-decoration-color: #800080; font-style: italic\">None</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'ai_primary_landmark_longitude'</span>: <span style=\"color: #800080; text-decoration-color: #800080; font-style: italic\">None</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'ai_primary_landmark_confidence'</span>: <span style=\"color: #800080; text-decoration-color: #800080; font-style: italic\">None</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'blur_hash'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'L56bVcRRIWMh.gVunlS4SMbsRRxr'</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'description_final'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'Woman exploring a forest'</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'image'</span>: <span style=\"font-weight: bold\">&lt;</span><span style=\"color: #ff00ff; text-decoration-color: #ff00ff; font-weight: bold\">PIL.JpegImagePlugin.JpegImageFile</span><span style=\"color: #000000; text-decoration-color: #000000\"> image </span><span style=\"color: #808000; text-decoration-color: #808000\">mode</span><span style=\"color: #000000; text-decoration-color: #000000\">=</span><span style=\"color: #800080; text-decoration-color: #800080\">RGB</span><span style=\"color: #000000; text-decoration-color: #000000\"> </span><span style=\"color: #808000; text-decoration-color: #808000\">size</span><span style=\"color: #000000; text-decoration-color: #000000\">=</span><span style=\"color: #800080; text-decoration-color: #800080\">64</span><span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0x427</span><span style=\"color: #000000; text-decoration-color: #000000\"> at </span><span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0x7F257129E950</span><span style=\"font-weight: bold\">&gt;</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'image_path_local'</span>: <span style=\"color: #008000; text-decoration-color: #008000\">'../data/raw/images/XMyPniM9LF0.jpg'</span>\n",
       "<span style=\"font-weight: bold\">}</span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m{\u001b[0m\n",
       "    \u001b[32m'photo_id'\u001b[0m: \u001b[32m'XMyPniM9LF0'\u001b[0m,\n",
       "    \u001b[32m'photo_url'\u001b[0m: \u001b[32m'https://unsplash.com/photos/XMyPniM9LF0'\u001b[0m,\n",
       "    \u001b[32m'photo_image_url'\u001b[0m: \u001b[32m'https://images.unsplash.com/uploads/14119492946973137ce46/f1f2ebf3'\u001b[0m,\n",
       "    \u001b[32m'photo_submitted_at'\u001b[0m: \u001b[32m'2014-09-29 00:08:38.594364'\u001b[0m,\n",
       "    \u001b[32m'photo_featured'\u001b[0m: \u001b[32m't'\u001b[0m,\n",
       "    \u001b[32m'photo_width'\u001b[0m: \u001b[1;36m4272\u001b[0m,\n",
       "    \u001b[32m'photo_height'\u001b[0m: \u001b[1;36m2848\u001b[0m,\n",
       "    \u001b[32m'photo_aspect_ratio'\u001b[0m: \u001b[1;36m1.5\u001b[0m,\n",
       "    \u001b[32m'photo_description'\u001b[0m: \u001b[32m'Woman exploring a forest'\u001b[0m,\n",
       "    \u001b[32m'photographer_username'\u001b[0m: \u001b[32m'michellespencer77'\u001b[0m,\n",
       "    \u001b[32m'photographer_first_name'\u001b[0m: \u001b[32m'Michelle'\u001b[0m,\n",
       "    \u001b[32m'photographer_last_name'\u001b[0m: \u001b[32m'Spencer'\u001b[0m,\n",
       "    \u001b[32m'exif_camera_make'\u001b[0m: \u001b[32m'Canon'\u001b[0m,\n",
       "    \u001b[32m'exif_camera_model'\u001b[0m: \u001b[32m'Canon EOS REBEL T3'\u001b[0m,\n",
       "    \u001b[32m'exif_iso'\u001b[0m: \u001b[1;36m400.0\u001b[0m,\n",
       "    \u001b[32m'exif_aperture_value'\u001b[0m: \u001b[32m'1.8'\u001b[0m,\n",
       "    \u001b[32m'exif_focal_length'\u001b[0m: \u001b[32m'50.0'\u001b[0m,\n",
       "    \u001b[32m'exif_exposure_time'\u001b[0m: \u001b[32m'1/100'\u001b[0m,\n",
       "    \u001b[32m'photo_location_name'\u001b[0m: \u001b[3;35mNone\u001b[0m,\n",
       "    \u001b[32m'photo_location_latitude'\u001b[0m: \u001b[3;35mNone\u001b[0m,\n",
       "    \u001b[32m'photo_location_longitude'\u001b[0m: \u001b[3;35mNone\u001b[0m,\n",
       "    \u001b[32m'photo_location_country'\u001b[0m: \u001b[3;35mNone\u001b[0m,\n",
       "    \u001b[32m'photo_location_city'\u001b[0m: \u001b[3;35mNone\u001b[0m,\n",
       "    \u001b[32m'stats_views'\u001b[0m: \u001b[1;36m2375421\u001b[0m,\n",
       "    \u001b[32m'stats_downloads'\u001b[0m: \u001b[1;36m6967\u001b[0m,\n",
       "    \u001b[32m'ai_description'\u001b[0m: \u001b[32m'woman walking in the middle of forest'\u001b[0m,\n",
       "    \u001b[32m'ai_primary_landmark_name'\u001b[0m: \u001b[3;35mNone\u001b[0m,\n",
       "    \u001b[32m'ai_primary_landmark_latitude'\u001b[0m: \u001b[3;35mNone\u001b[0m,\n",
       "    \u001b[32m'ai_primary_landmark_longitude'\u001b[0m: \u001b[3;35mNone\u001b[0m,\n",
       "    \u001b[32m'ai_primary_landmark_confidence'\u001b[0m: \u001b[3;35mNone\u001b[0m,\n",
       "    \u001b[32m'blur_hash'\u001b[0m: \u001b[32m'L56bVcRRIWMh.gVunlS4SMbsRRxr'\u001b[0m,\n",
       "    \u001b[32m'description_final'\u001b[0m: \u001b[32m'Woman exploring a forest'\u001b[0m,\n",
       "    \u001b[32m'image'\u001b[0m: \u001b[1m<\u001b[0m\u001b[1;95mPIL.JpegImagePlugin.JpegImageFile\u001b[0m\u001b[39m image \u001b[0m\u001b[33mmode\u001b[0m\u001b[39m=\u001b[0m\u001b[35mRGB\u001b[0m\u001b[39m \u001b[0m\u001b[33msize\u001b[0m\u001b[39m=\u001b[0m\u001b[35m64\u001b[0m\u001b[1;36m0x427\u001b[0m\u001b[39m at \u001b[0m\u001b[1;36m0x7F257129E950\u001b[0m\u001b[1m>\u001b[0m,\n",
       "    \u001b[32m'image_path_local'\u001b[0m: \u001b[32m'../data/raw/images/XMyPniM9LF0.jpg'\u001b[0m\n",
       "\u001b[1m}\u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "rich.print ( dset[0] )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b79a0ebd-ca41-4a8d-a700-41003fe13d27",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "21d8bf82-6d50-41f2-ad17-54a0d1ff7122",
   "metadata": {},
   "source": [
    "## Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8e26e171-8c4c-495d-b037-c5f6eb33b468",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "58510833-873a-46d4-a364-715c4c3a53dd",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-26T12:49:08.966068Z",
     "iopub.status.busy": "2023-04-26T12:49:08.965841Z",
     "iopub.status.idle": "2023-04-26T12:49:08.969392Z",
     "shell.execute_reply": "2023-04-26T12:49:08.968664Z",
     "shell.execute_reply.started": "2023-04-26T12:49:08.966047Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "model_name = 'sentence-transformers/all-MiniLM-L6-v2'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "ef7e8dff-3805-4b25-9bae-6ce925e00c6f",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-26T12:49:08.970668Z",
     "iopub.status.busy": "2023-04-26T12:49:08.970407Z",
     "iopub.status.idle": "2023-04-26T12:49:13.872458Z",
     "shell.execute_reply": "2023-04-26T12:49:13.871478Z",
     "shell.execute_reply.started": "2023-04-26T12:49:08.970631Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "SentenceTransformer(\n",
       "  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel \n",
       "  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False})\n",
       "  (2): Normalize()\n",
       ")"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\n",
    "# Initialize retriever with SentenceTransformer model \n",
    "model = SentenceTransformer(model_name)\n",
    "model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4488d1d2-1450-41ac-a8bf-704435235daf",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "129a663f-223a-43ef-97b6-62d7af0ee1f3",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Tokenizer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fe36d3e9-4b5a-443c-8b4d-d1b8cc1579e1",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "689f4f53-0b39-4237-bb22-581ac0a5e50f",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "e84a0052-9f25-4b34-8678-aaa2c964ccc6",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-26T12:56:07.546489Z",
     "iopub.status.busy": "2023-04-26T12:56:07.545580Z",
     "iopub.status.idle": "2023-04-26T12:56:07.550056Z",
     "shell.execute_reply": "2023-04-26T12:56:07.549150Z",
     "shell.execute_reply.started": "2023-04-26T12:56:07.546455Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "text = \"A man and his pet dog Perrito playdingd catch in the beach.\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "id": "db0d360b-58e6-4469-a10a-42273efccbfc",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-26T12:56:07.858013Z",
     "iopub.status.busy": "2023-04-26T12:56:07.857273Z",
     "iopub.status.idle": "2023-04-26T12:56:07.899753Z",
     "shell.execute_reply": "2023-04-26T12:56:07.899044Z",
     "shell.execute_reply.started": "2023-04-26T12:56:07.857975Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\">{</span>\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'input_ids'</span>: <span style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\">tensor</span><span style=\"font-weight: bold\">([</span>  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">101</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1037</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">2158</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1998</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">2010</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">9004</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">3899</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">2566</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">28414</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">2377</span>,\n",
       "         <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">4667</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">2094</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">4608</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1999</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1996</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">3509</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1012</span>,   <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">102</span><span style=\"font-weight: bold\">]</span>,\n",
       "       <span style=\"color: #808000; text-decoration-color: #808000\">device</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'cuda:0'</span><span style=\"font-weight: bold\">)</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'token_type_ids'</span>: <span style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\">tensor</span><span style=\"font-weight: bold\">([</span><span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0</span><span style=\"font-weight: bold\">]</span>, <span style=\"color: #808000; text-decoration-color: #808000\">device</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'cuda:0'</span><span style=\"font-weight: bold\">)</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'attention_mask'</span>: <span style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\">tensor</span><span style=\"font-weight: bold\">([</span><span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1</span><span style=\"font-weight: bold\">]</span>, <span style=\"color: #808000; text-decoration-color: #808000\">device</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'cuda:0'</span><span style=\"font-weight: bold\">)</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'token_embeddings'</span>: <span style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\">tensor</span><span style=\"font-weight: bold\">([[</span><span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-0.0677</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0.0513</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0.4768</span>,  <span style=\"color: #808000; text-decoration-color: #808000\">...</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0.3576</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-0.3906</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-0.1148</span><span style=\"font-weight: bold\">]</span>,\n",
       "        <span style=\"font-weight: bold\">[</span><span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-0.1968</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-0.0721</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-0.3324</span>,  <span style=\"color: #808000; text-decoration-color: #808000\">...</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0.5298</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-0.2774</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0.5287</span><span style=\"font-weight: bold\">]</span>,\n",
       "        <span style=\"font-weight: bold\">[</span> <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0.0858</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0.4833</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-0.3329</span>,  <span style=\"color: #808000; text-decoration-color: #808000\">...</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-0.3483</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-0.4282</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-0.2897</span><span style=\"font-weight: bold\">]</span>,\n",
       "        <span style=\"color: #808000; text-decoration-color: #808000\">...</span>,\n",
       "        <span style=\"font-weight: bold\">[</span> <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0.7096</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0.1959</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0.1379</span>,  <span style=\"color: #808000; text-decoration-color: #808000\">...</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0.0101</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-0.1908</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-0.1463</span><span style=\"font-weight: bold\">]</span>,\n",
       "        <span style=\"font-weight: bold\">[</span> <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0.0943</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0.1842</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0.1608</span>,  <span style=\"color: #808000; text-decoration-color: #808000\">...</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0.4050</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-0.3372</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0.2188</span><span style=\"font-weight: bold\">]</span>,\n",
       "        <span style=\"font-weight: bold\">[</span> <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0.1566</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0.1167</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0.1975</span>,  <span style=\"color: #808000; text-decoration-color: #808000\">...</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0.3038</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-0.2227</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">0.2381</span><span style=\"font-weight: bold\">]]</span>,\n",
       "       <span style=\"color: #808000; text-decoration-color: #808000\">device</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'cuda:0'</span><span style=\"font-weight: bold\">)</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'sentence_embedding'</span>: <span style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\">tensor</span><span style=\"font-weight: bold\">([</span> <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">2.7245e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">8.5115e-04</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">4.1074e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-6.1840e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-4.6796e-02</span>,\n",
       "        <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-3.2752e-03</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">6.4249e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-3.3732e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">3.3450e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1.3065e-02</span>,\n",
       "         <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">8.5493e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-4.2203e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">2.4165e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">5.2184e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">7.6807e-02</span>,\n",
       "         <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">3.9865e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-5.7506e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">2.4284e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">8.2526e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-3.1512e-02</span>,\n",
       "        <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-6.5045e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">8.3287e-03</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-1.0168e-01</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-5.3761e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-1.8875e-01</span>,\n",
       "        <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-3.0703e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">7.5860e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-2.2065e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-8.8094e-03</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-1.9613e-02</span>,\n",
       "        <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-1.9956e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-1.8503e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">6.7821e-03</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1.7351e-03</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-4.2387e-02</span>,\n",
       "        <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-3.5774e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-4.4546e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-3.3200e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-1.3204e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-2.5218e-04</span>,\n",
       "        <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-2.1067e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-1.3864e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">8.8424e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">2.3969e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-4.1921e-02</span>,\n",
       "        <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-1.2925e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-1.5744e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-4.8291e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">8.3471e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-3.5383e-02</span>,\n",
       "        <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-4.0611e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">2.0836e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-4.1337e-03</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-2.6046e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-7.0935e-02</span>,\n",
       "        <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-4.3526e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-8.0206e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">3.2333e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-8.0339e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-1.4097e-02</span>,\n",
       "         <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">5.7811e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">2.1070e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">4.7009e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1.0136e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">5.0523e-02</span>,\n",
       "        <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-4.3152e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1.2582e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">2.0966e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-1.4002e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-2.4897e-02</span>,\n",
       "         <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1.5817e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-1.2991e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">7.8685e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-5.7955e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-1.9291e-02</span>,\n",
       "         <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">2.6861e-03</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-7.0163e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">3.8303e-03</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1.1840e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-6.3685e-02</span>,\n",
       "        <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-5.4816e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1.6686e-03</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-7.1812e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1.8636e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">8.1678e-02</span>,\n",
       "         <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">3.1124e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-2.3793e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1.9062e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1.8774e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">3.5100e-02</span>,\n",
       "        <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-8.2608e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-6.7733e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-5.0751e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-3.9201e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-6.1961e-02</span>,\n",
       "         <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">4.2451e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-3.8355e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">6.1236e-03</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-2.9451e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">4.0979e-03</span>,\n",
       "         <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">3.8161e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">6.1894e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-4.5156e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-8.8139e-04</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1.5279e-01</span>,\n",
       "         <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">4.4970e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">3.9380e-03</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">5.9288e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">5.6811e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">6.3213e-02</span>,\n",
       "        <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-5.9248e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">8.1122e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-1.4986e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-3.3264e-03</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">5.6281e-02</span>,\n",
       "         <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1.0469e-01</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-2.2121e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-6.2191e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-1.7965e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-1.0685e-01</span>,\n",
       "         <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1.1008e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">3.6424e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-3.8871e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">3.3848e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">5.2330e-02</span>,\n",
       "         <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">3.2203e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">3.8796e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-3.2964e-33</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">2.6144e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-8.3933e-03</span>,\n",
       "        <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-4.7975e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1.7831e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-7.7705e-03</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">2.3873e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-5.4458e-02</span>,\n",
       "        <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-3.1801e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-3.4542e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-2.6412e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-2.8960e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-5.7436e-03</span>,\n",
       "        <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-3.6600e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1.8724e-04</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">4.3041e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-7.2709e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">3.3226e-02</span>,\n",
       "        <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-6.4818e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">3.5052e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-3.7997e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-2.1842e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-7.1258e-03</span>,\n",
       "        <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-4.4505e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-4.2587e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-1.1228e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">8.2499e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-6.4990e-03</span>,\n",
       "        <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-4.5247e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">3.6797e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-9.9252e-03</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-2.2450e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-9.3727e-03</span>,\n",
       "         <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">3.2621e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-3.0249e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-6.1823e-03</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-2.2296e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-6.8490e-03</span>,\n",
       "        <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-4.0892e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-3.6306e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">5.1262e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1.2915e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">7.6723e-03</span>,\n",
       "         <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1.4748e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-2.4131e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-8.1759e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-1.3676e-01</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-2.8145e-02</span>,\n",
       "         <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">6.4891e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1.8963e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-5.3716e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1.5972e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-2.5845e-02</span>,\n",
       "         <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">4.5339e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-4.3962e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-4.5941e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">3.5499e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">5.3470e-02</span>,\n",
       "        <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-1.4657e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-6.0126e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1.2750e-01</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1.0249e-01</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">6.4104e-03</span>,\n",
       "         <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">5.3865e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">3.9636e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">2.3959e-03</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-5.0817e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">4.1301e-02</span>,\n",
       "         <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">5.6234e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">5.6352e-03</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1.2783e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">7.5478e-03</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-4.4354e-03</span>,\n",
       "         <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1.1062e-01</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-4.0939e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">3.5345e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-1.9199e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1.4538e-02</span>,\n",
       "         <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">2.5399e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">4.6266e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-3.5740e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1.2718e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1.9636e-02</span>,\n",
       "         <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1.0668e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1.9186e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-1.1912e-01</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1.7670e-01</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1.0298e-02</span>,\n",
       "         <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1.0925e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">6.0375e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">9.2171e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-5.0339e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">2.3069e-02</span>,\n",
       "         <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">5.3641e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-4.9093e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">5.2999e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-1.1273e-33</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-9.4553e-02</span>,\n",
       "         <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">4.0063e-03</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">2.4540e-03</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-9.4763e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">5.6738e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-8.9051e-03</span>,\n",
       "        <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-1.5367e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">3.7929e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-3.1128e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-5.5165e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-2.0374e-01</span>,\n",
       "         <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1.0024e-01</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">8.9946e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-8.1773e-03</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-4.6732e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">5.4862e-02</span>,\n",
       "        <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-4.9572e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1.6269e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">5.5046e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-4.6725e-03</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-2.6217e-02</span>,\n",
       "        <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-2.7294e-03</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1.0804e-01</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">4.4482e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-3.9526e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">4.1552e-02</span>,\n",
       "         <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">4.2067e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">6.9128e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-7.0026e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-4.0075e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-4.2712e-04</span>,\n",
       "         <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">2.8174e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">4.9838e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-9.1041e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-6.0918e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-5.0477e-02</span>,\n",
       "        <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-8.3952e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-1.4577e-03</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-6.7692e-03</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">4.1205e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">3.3702e-02</span>,\n",
       "         <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">2.6134e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-9.3400e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-1.7563e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-5.4995e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1.2440e-01</span>,\n",
       "        <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-8.1775e-03</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-1.0323e-01</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">2.4818e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-6.4749e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-3.5071e-02</span>,\n",
       "         <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">3.1912e-04</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-4.1279e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-1.4959e-01</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-2.2634e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">5.8337e-02</span>,\n",
       "         <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">7.4144e-03</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">6.4620e-03</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">3.8110e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">4.3539e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-5.4276e-02</span>,\n",
       "         <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">8.7545e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-1.4871e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">9.6427e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-2.2182e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">4.9424e-02</span>,\n",
       "        <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-1.6440e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-4.0981e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">2.6056e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-5.1585e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-3.1748e-02</span>,\n",
       "         <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1.0050e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-1.0158e-01</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-7.4725e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">5.3766e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">5.9533e-02</span>,\n",
       "        <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-8.9224e-03</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">3.0551e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">8.3009e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">2.2091e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-1.6830e-02</span>,\n",
       "        <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-1.2798e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-5.4892e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">2.0980e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-1.0143e-01</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">8.2567e-03</span>,\n",
       "         <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">5.7449e-03</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1.2578e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-4.7935e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-2.4858e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">3.1339e-02</span>,\n",
       "         <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">9.7304e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-2.1196e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-2.9524e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-3.0847e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-1.8560e-08</span>,\n",
       "        <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-5.2432e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-7.1539e-03</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">2.7848e-03</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1.3505e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-1.8513e-03</span>,\n",
       "         <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">3.4044e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-5.6208e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-6.3033e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">2.5686e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">6.0419e-02</span>,\n",
       "        <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-2.5544e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-5.4425e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-6.2813e-03</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">2.0601e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-1.3528e-02</span>,\n",
       "         <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">2.4703e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">7.7846e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">2.3287e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-3.3436e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">6.4842e-02</span>,\n",
       "        <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-4.4627e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1.1172e-01</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1.5232e-03</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-5.2231e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-4.1692e-02</span>,\n",
       "         <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">6.6046e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1.7801e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">2.7188e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-1.3282e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">6.9399e-02</span>,\n",
       "        <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-2.3109e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-1.5641e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-4.6476e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-3.3069e-04</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">7.5727e-02</span>,\n",
       "        <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-1.3049e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">8.1173e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-6.3903e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">4.9146e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">3.6656e-02</span>,\n",
       "        <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-3.1578e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">7.6392e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">7.1512e-03</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">9.1036e-04</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">3.4462e-02</span>,\n",
       "         <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">7.1568e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1.2789e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1.5305e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">1.0297e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">3.0290e-02</span>,\n",
       "        <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-4.6647e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-1.1118e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-1.6685e-03</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-7.5788e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">2.5742e-02</span>,\n",
       "        <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-3.1140e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-2.8856e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-6.4451e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">4.0715e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-3.2946e-02</span>,\n",
       "        <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-1.1134e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">5.0589e-02</span>, <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">-1.3195e-02</span>,  <span style=\"color: #008080; text-decoration-color: #008080; font-weight: bold\">3.2110e-02</span><span style=\"font-weight: bold\">]</span>, <span style=\"color: #808000; text-decoration-color: #808000\">device</span>=<span style=\"color: #008000; text-decoration-color: #008000\">'cuda:0'</span><span style=\"font-weight: bold\">)</span>\n",
       "<span style=\"font-weight: bold\">}</span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m{\u001b[0m\n",
       "    \u001b[32m'input_ids'\u001b[0m: \u001b[1;35mtensor\u001b[0m\u001b[1m(\u001b[0m\u001b[1m[\u001b[0m  \u001b[1;36m101\u001b[0m,  \u001b[1;36m1037\u001b[0m,  \u001b[1;36m2158\u001b[0m,  \u001b[1;36m1998\u001b[0m,  \u001b[1;36m2010\u001b[0m,  \u001b[1;36m9004\u001b[0m,  \u001b[1;36m3899\u001b[0m,  \u001b[1;36m2566\u001b[0m, \u001b[1;36m28414\u001b[0m,  \u001b[1;36m2377\u001b[0m,\n",
       "         \u001b[1;36m4667\u001b[0m,  \u001b[1;36m2094\u001b[0m,  \u001b[1;36m4608\u001b[0m,  \u001b[1;36m1999\u001b[0m,  \u001b[1;36m1996\u001b[0m,  \u001b[1;36m3509\u001b[0m,  \u001b[1;36m1012\u001b[0m,   \u001b[1;36m102\u001b[0m\u001b[1m]\u001b[0m,\n",
       "       \u001b[33mdevice\u001b[0m=\u001b[32m'cuda:0'\u001b[0m\u001b[1m)\u001b[0m,\n",
       "    \u001b[32m'token_type_ids'\u001b[0m: \u001b[1;35mtensor\u001b[0m\u001b[1m(\u001b[0m\u001b[1m[\u001b[0m\u001b[1;36m0\u001b[0m, \u001b[1;36m0\u001b[0m, \u001b[1;36m0\u001b[0m, \u001b[1;36m0\u001b[0m, \u001b[1;36m0\u001b[0m, \u001b[1;36m0\u001b[0m, \u001b[1;36m0\u001b[0m, \u001b[1;36m0\u001b[0m, \u001b[1;36m0\u001b[0m, \u001b[1;36m0\u001b[0m, \u001b[1;36m0\u001b[0m, \u001b[1;36m0\u001b[0m, \u001b[1;36m0\u001b[0m, \u001b[1;36m0\u001b[0m, \u001b[1;36m0\u001b[0m, \u001b[1;36m0\u001b[0m, \u001b[1;36m0\u001b[0m, \u001b[1;36m0\u001b[0m\u001b[1m]\u001b[0m, \u001b[33mdevice\u001b[0m=\u001b[32m'cuda:0'\u001b[0m\u001b[1m)\u001b[0m,\n",
       "    \u001b[32m'attention_mask'\u001b[0m: \u001b[1;35mtensor\u001b[0m\u001b[1m(\u001b[0m\u001b[1m[\u001b[0m\u001b[1;36m1\u001b[0m, \u001b[1;36m1\u001b[0m, \u001b[1;36m1\u001b[0m, \u001b[1;36m1\u001b[0m, \u001b[1;36m1\u001b[0m, \u001b[1;36m1\u001b[0m, \u001b[1;36m1\u001b[0m, \u001b[1;36m1\u001b[0m, \u001b[1;36m1\u001b[0m, \u001b[1;36m1\u001b[0m, \u001b[1;36m1\u001b[0m, \u001b[1;36m1\u001b[0m, \u001b[1;36m1\u001b[0m, \u001b[1;36m1\u001b[0m, \u001b[1;36m1\u001b[0m, \u001b[1;36m1\u001b[0m, \u001b[1;36m1\u001b[0m, \u001b[1;36m1\u001b[0m\u001b[1m]\u001b[0m, \u001b[33mdevice\u001b[0m=\u001b[32m'cuda:0'\u001b[0m\u001b[1m)\u001b[0m,\n",
       "    \u001b[32m'token_embeddings'\u001b[0m: \u001b[1;35mtensor\u001b[0m\u001b[1m(\u001b[0m\u001b[1m[\u001b[0m\u001b[1m[\u001b[0m\u001b[1;36m-0.0677\u001b[0m,  \u001b[1;36m0.0513\u001b[0m,  \u001b[1;36m0.4768\u001b[0m,  \u001b[33m...\u001b[0m,  \u001b[1;36m0.3576\u001b[0m, \u001b[1;36m-0.3906\u001b[0m, \u001b[1;36m-0.1148\u001b[0m\u001b[1m]\u001b[0m,\n",
       "        \u001b[1m[\u001b[0m\u001b[1;36m-0.1968\u001b[0m, \u001b[1;36m-0.0721\u001b[0m, \u001b[1;36m-0.3324\u001b[0m,  \u001b[33m...\u001b[0m,  \u001b[1;36m0.5298\u001b[0m, \u001b[1;36m-0.2774\u001b[0m,  \u001b[1;36m0.5287\u001b[0m\u001b[1m]\u001b[0m,\n",
       "        \u001b[1m[\u001b[0m \u001b[1;36m0.0858\u001b[0m,  \u001b[1;36m0.4833\u001b[0m, \u001b[1;36m-0.3329\u001b[0m,  \u001b[33m...\u001b[0m, \u001b[1;36m-0.3483\u001b[0m, \u001b[1;36m-0.4282\u001b[0m, \u001b[1;36m-0.2897\u001b[0m\u001b[1m]\u001b[0m,\n",
       "        \u001b[33m...\u001b[0m,\n",
       "        \u001b[1m[\u001b[0m \u001b[1;36m0.7096\u001b[0m,  \u001b[1;36m0.1959\u001b[0m,  \u001b[1;36m0.1379\u001b[0m,  \u001b[33m...\u001b[0m,  \u001b[1;36m0.0101\u001b[0m, \u001b[1;36m-0.1908\u001b[0m, \u001b[1;36m-0.1463\u001b[0m\u001b[1m]\u001b[0m,\n",
       "        \u001b[1m[\u001b[0m \u001b[1;36m0.0943\u001b[0m,  \u001b[1;36m0.1842\u001b[0m,  \u001b[1;36m0.1608\u001b[0m,  \u001b[33m...\u001b[0m,  \u001b[1;36m0.4050\u001b[0m, \u001b[1;36m-0.3372\u001b[0m,  \u001b[1;36m0.2188\u001b[0m\u001b[1m]\u001b[0m,\n",
       "        \u001b[1m[\u001b[0m \u001b[1;36m0.1566\u001b[0m,  \u001b[1;36m0.1167\u001b[0m,  \u001b[1;36m0.1975\u001b[0m,  \u001b[33m...\u001b[0m,  \u001b[1;36m0.3038\u001b[0m, \u001b[1;36m-0.2227\u001b[0m,  \u001b[1;36m0.2381\u001b[0m\u001b[1m]\u001b[0m\u001b[1m]\u001b[0m,\n",
       "       \u001b[33mdevice\u001b[0m=\u001b[32m'cuda:0'\u001b[0m\u001b[1m)\u001b[0m,\n",
       "    \u001b[32m'sentence_embedding'\u001b[0m: \u001b[1;35mtensor\u001b[0m\u001b[1m(\u001b[0m\u001b[1m[\u001b[0m \u001b[1;36m2.7245e-02\u001b[0m,  \u001b[1;36m8.5115e-04\u001b[0m,  \u001b[1;36m4.1074e-02\u001b[0m, \u001b[1;36m-6.1840e-02\u001b[0m, \u001b[1;36m-4.6796e-02\u001b[0m,\n",
       "        \u001b[1;36m-3.2752e-03\u001b[0m,  \u001b[1;36m6.4249e-02\u001b[0m, \u001b[1;36m-3.3732e-02\u001b[0m,  \u001b[1;36m3.3450e-02\u001b[0m,  \u001b[1;36m1.3065e-02\u001b[0m,\n",
       "         \u001b[1;36m8.5493e-02\u001b[0m, \u001b[1;36m-4.2203e-02\u001b[0m,  \u001b[1;36m2.4165e-02\u001b[0m,  \u001b[1;36m5.2184e-02\u001b[0m,  \u001b[1;36m7.6807e-02\u001b[0m,\n",
       "         \u001b[1;36m3.9865e-02\u001b[0m, \u001b[1;36m-5.7506e-02\u001b[0m,  \u001b[1;36m2.4284e-02\u001b[0m,  \u001b[1;36m8.2526e-02\u001b[0m, \u001b[1;36m-3.1512e-02\u001b[0m,\n",
       "        \u001b[1;36m-6.5045e-02\u001b[0m,  \u001b[1;36m8.3287e-03\u001b[0m, \u001b[1;36m-1.0168e-01\u001b[0m, \u001b[1;36m-5.3761e-02\u001b[0m, \u001b[1;36m-1.8875e-01\u001b[0m,\n",
       "        \u001b[1;36m-3.0703e-02\u001b[0m,  \u001b[1;36m7.5860e-02\u001b[0m, \u001b[1;36m-2.2065e-02\u001b[0m, \u001b[1;36m-8.8094e-03\u001b[0m, \u001b[1;36m-1.9613e-02\u001b[0m,\n",
       "        \u001b[1;36m-1.9956e-02\u001b[0m, \u001b[1;36m-1.8503e-02\u001b[0m,  \u001b[1;36m6.7821e-03\u001b[0m,  \u001b[1;36m1.7351e-03\u001b[0m, \u001b[1;36m-4.2387e-02\u001b[0m,\n",
       "        \u001b[1;36m-3.5774e-02\u001b[0m, \u001b[1;36m-4.4546e-02\u001b[0m, \u001b[1;36m-3.3200e-02\u001b[0m, \u001b[1;36m-1.3204e-02\u001b[0m, \u001b[1;36m-2.5218e-04\u001b[0m,\n",
       "        \u001b[1;36m-2.1067e-02\u001b[0m, \u001b[1;36m-1.3864e-02\u001b[0m,  \u001b[1;36m8.8424e-02\u001b[0m,  \u001b[1;36m2.3969e-02\u001b[0m, \u001b[1;36m-4.1921e-02\u001b[0m,\n",
       "        \u001b[1;36m-1.2925e-02\u001b[0m, \u001b[1;36m-1.5744e-02\u001b[0m, \u001b[1;36m-4.8291e-02\u001b[0m,  \u001b[1;36m8.3471e-02\u001b[0m, \u001b[1;36m-3.5383e-02\u001b[0m,\n",
       "        \u001b[1;36m-4.0611e-02\u001b[0m,  \u001b[1;36m2.0836e-02\u001b[0m, \u001b[1;36m-4.1337e-03\u001b[0m, \u001b[1;36m-2.6046e-02\u001b[0m, \u001b[1;36m-7.0935e-02\u001b[0m,\n",
       "        \u001b[1;36m-4.3526e-02\u001b[0m, \u001b[1;36m-8.0206e-02\u001b[0m,  \u001b[1;36m3.2333e-02\u001b[0m, \u001b[1;36m-8.0339e-02\u001b[0m, \u001b[1;36m-1.4097e-02\u001b[0m,\n",
       "         \u001b[1;36m5.7811e-02\u001b[0m,  \u001b[1;36m2.1070e-02\u001b[0m,  \u001b[1;36m4.7009e-02\u001b[0m,  \u001b[1;36m1.0136e-02\u001b[0m,  \u001b[1;36m5.0523e-02\u001b[0m,\n",
       "        \u001b[1;36m-4.3152e-02\u001b[0m,  \u001b[1;36m1.2582e-02\u001b[0m,  \u001b[1;36m2.0966e-02\u001b[0m, \u001b[1;36m-1.4002e-02\u001b[0m, \u001b[1;36m-2.4897e-02\u001b[0m,\n",
       "         \u001b[1;36m1.5817e-02\u001b[0m, \u001b[1;36m-1.2991e-02\u001b[0m,  \u001b[1;36m7.8685e-02\u001b[0m, \u001b[1;36m-5.7955e-02\u001b[0m, \u001b[1;36m-1.9291e-02\u001b[0m,\n",
       "         \u001b[1;36m2.6861e-03\u001b[0m, \u001b[1;36m-7.0163e-02\u001b[0m,  \u001b[1;36m3.8303e-03\u001b[0m,  \u001b[1;36m1.1840e-02\u001b[0m, \u001b[1;36m-6.3685e-02\u001b[0m,\n",
       "        \u001b[1;36m-5.4816e-02\u001b[0m,  \u001b[1;36m1.6686e-03\u001b[0m, \u001b[1;36m-7.1812e-02\u001b[0m,  \u001b[1;36m1.8636e-02\u001b[0m,  \u001b[1;36m8.1678e-02\u001b[0m,\n",
       "         \u001b[1;36m3.1124e-02\u001b[0m, \u001b[1;36m-2.3793e-02\u001b[0m,  \u001b[1;36m1.9062e-02\u001b[0m,  \u001b[1;36m1.8774e-02\u001b[0m,  \u001b[1;36m3.5100e-02\u001b[0m,\n",
       "        \u001b[1;36m-8.2608e-02\u001b[0m, \u001b[1;36m-6.7733e-02\u001b[0m, \u001b[1;36m-5.0751e-02\u001b[0m, \u001b[1;36m-3.9201e-02\u001b[0m, \u001b[1;36m-6.1961e-02\u001b[0m,\n",
       "         \u001b[1;36m4.2451e-02\u001b[0m, \u001b[1;36m-3.8355e-02\u001b[0m,  \u001b[1;36m6.1236e-03\u001b[0m, \u001b[1;36m-2.9451e-02\u001b[0m,  \u001b[1;36m4.0979e-03\u001b[0m,\n",
       "         \u001b[1;36m3.8161e-02\u001b[0m,  \u001b[1;36m6.1894e-02\u001b[0m, \u001b[1;36m-4.5156e-02\u001b[0m, \u001b[1;36m-8.8139e-04\u001b[0m,  \u001b[1;36m1.5279e-01\u001b[0m,\n",
       "         \u001b[1;36m4.4970e-02\u001b[0m,  \u001b[1;36m3.9380e-03\u001b[0m,  \u001b[1;36m5.9288e-02\u001b[0m,  \u001b[1;36m5.6811e-02\u001b[0m,  \u001b[1;36m6.3213e-02\u001b[0m,\n",
       "        \u001b[1;36m-5.9248e-02\u001b[0m,  \u001b[1;36m8.1122e-02\u001b[0m, \u001b[1;36m-1.4986e-02\u001b[0m, \u001b[1;36m-3.3264e-03\u001b[0m,  \u001b[1;36m5.6281e-02\u001b[0m,\n",
       "         \u001b[1;36m1.0469e-01\u001b[0m, \u001b[1;36m-2.2121e-02\u001b[0m, \u001b[1;36m-6.2191e-02\u001b[0m, \u001b[1;36m-1.7965e-02\u001b[0m, \u001b[1;36m-1.0685e-01\u001b[0m,\n",
       "         \u001b[1;36m1.1008e-02\u001b[0m,  \u001b[1;36m3.6424e-02\u001b[0m, \u001b[1;36m-3.8871e-02\u001b[0m,  \u001b[1;36m3.3848e-02\u001b[0m,  \u001b[1;36m5.2330e-02\u001b[0m,\n",
       "         \u001b[1;36m3.2203e-02\u001b[0m,  \u001b[1;36m3.8796e-02\u001b[0m, \u001b[1;36m-3.2964e-33\u001b[0m,  \u001b[1;36m2.6144e-02\u001b[0m, \u001b[1;36m-8.3933e-03\u001b[0m,\n",
       "        \u001b[1;36m-4.7975e-02\u001b[0m,  \u001b[1;36m1.7831e-02\u001b[0m, \u001b[1;36m-7.7705e-03\u001b[0m,  \u001b[1;36m2.3873e-02\u001b[0m, \u001b[1;36m-5.4458e-02\u001b[0m,\n",
       "        \u001b[1;36m-3.1801e-02\u001b[0m, \u001b[1;36m-3.4542e-02\u001b[0m, \u001b[1;36m-2.6412e-02\u001b[0m, \u001b[1;36m-2.8960e-02\u001b[0m, \u001b[1;36m-5.7436e-03\u001b[0m,\n",
       "        \u001b[1;36m-3.6600e-02\u001b[0m,  \u001b[1;36m1.8724e-04\u001b[0m,  \u001b[1;36m4.3041e-02\u001b[0m, \u001b[1;36m-7.2709e-02\u001b[0m,  \u001b[1;36m3.3226e-02\u001b[0m,\n",
       "        \u001b[1;36m-6.4818e-02\u001b[0m,  \u001b[1;36m3.5052e-02\u001b[0m, \u001b[1;36m-3.7997e-02\u001b[0m, \u001b[1;36m-2.1842e-02\u001b[0m, \u001b[1;36m-7.1258e-03\u001b[0m,\n",
       "        \u001b[1;36m-4.4505e-02\u001b[0m, \u001b[1;36m-4.2587e-02\u001b[0m, \u001b[1;36m-1.1228e-02\u001b[0m,  \u001b[1;36m8.2499e-02\u001b[0m, \u001b[1;36m-6.4990e-03\u001b[0m,\n",
       "        \u001b[1;36m-4.5247e-02\u001b[0m,  \u001b[1;36m3.6797e-02\u001b[0m, \u001b[1;36m-9.9252e-03\u001b[0m, \u001b[1;36m-2.2450e-02\u001b[0m, \u001b[1;36m-9.3727e-03\u001b[0m,\n",
       "         \u001b[1;36m3.2621e-02\u001b[0m, \u001b[1;36m-3.0249e-02\u001b[0m, \u001b[1;36m-6.1823e-03\u001b[0m, \u001b[1;36m-2.2296e-02\u001b[0m, \u001b[1;36m-6.8490e-03\u001b[0m,\n",
       "        \u001b[1;36m-4.0892e-02\u001b[0m, \u001b[1;36m-3.6306e-02\u001b[0m,  \u001b[1;36m5.1262e-02\u001b[0m,  \u001b[1;36m1.2915e-02\u001b[0m,  \u001b[1;36m7.6723e-03\u001b[0m,\n",
       "         \u001b[1;36m1.4748e-02\u001b[0m, \u001b[1;36m-2.4131e-02\u001b[0m, \u001b[1;36m-8.1759e-02\u001b[0m, \u001b[1;36m-1.3676e-01\u001b[0m, \u001b[1;36m-2.8145e-02\u001b[0m,\n",
       "         \u001b[1;36m6.4891e-02\u001b[0m,  \u001b[1;36m1.8963e-02\u001b[0m, \u001b[1;36m-5.3716e-02\u001b[0m,  \u001b[1;36m1.5972e-02\u001b[0m, \u001b[1;36m-2.5845e-02\u001b[0m,\n",
       "         \u001b[1;36m4.5339e-02\u001b[0m, \u001b[1;36m-4.3962e-02\u001b[0m, \u001b[1;36m-4.5941e-02\u001b[0m,  \u001b[1;36m3.5499e-02\u001b[0m,  \u001b[1;36m5.3470e-02\u001b[0m,\n",
       "        \u001b[1;36m-1.4657e-02\u001b[0m, \u001b[1;36m-6.0126e-02\u001b[0m,  \u001b[1;36m1.2750e-01\u001b[0m,  \u001b[1;36m1.0249e-01\u001b[0m,  \u001b[1;36m6.4104e-03\u001b[0m,\n",
       "         \u001b[1;36m5.3865e-02\u001b[0m,  \u001b[1;36m3.9636e-02\u001b[0m,  \u001b[1;36m2.3959e-03\u001b[0m, \u001b[1;36m-5.0817e-02\u001b[0m,  \u001b[1;36m4.1301e-02\u001b[0m,\n",
       "         \u001b[1;36m5.6234e-02\u001b[0m,  \u001b[1;36m5.6352e-03\u001b[0m,  \u001b[1;36m1.2783e-02\u001b[0m,  \u001b[1;36m7.5478e-03\u001b[0m, \u001b[1;36m-4.4354e-03\u001b[0m,\n",
       "         \u001b[1;36m1.1062e-01\u001b[0m, \u001b[1;36m-4.0939e-02\u001b[0m,  \u001b[1;36m3.5345e-02\u001b[0m, \u001b[1;36m-1.9199e-02\u001b[0m,  \u001b[1;36m1.4538e-02\u001b[0m,\n",
       "         \u001b[1;36m2.5399e-02\u001b[0m,  \u001b[1;36m4.6266e-02\u001b[0m, \u001b[1;36m-3.5740e-02\u001b[0m,  \u001b[1;36m1.2718e-02\u001b[0m,  \u001b[1;36m1.9636e-02\u001b[0m,\n",
       "         \u001b[1;36m1.0668e-02\u001b[0m,  \u001b[1;36m1.9186e-02\u001b[0m, \u001b[1;36m-1.1912e-01\u001b[0m,  \u001b[1;36m1.7670e-01\u001b[0m,  \u001b[1;36m1.0298e-02\u001b[0m,\n",
       "         \u001b[1;36m1.0925e-02\u001b[0m,  \u001b[1;36m6.0375e-02\u001b[0m,  \u001b[1;36m9.2171e-02\u001b[0m, \u001b[1;36m-5.0339e-02\u001b[0m,  \u001b[1;36m2.3069e-02\u001b[0m,\n",
       "         \u001b[1;36m5.3641e-02\u001b[0m, \u001b[1;36m-4.9093e-02\u001b[0m,  \u001b[1;36m5.2999e-02\u001b[0m, \u001b[1;36m-1.1273e-33\u001b[0m, \u001b[1;36m-9.4553e-02\u001b[0m,\n",
       "         \u001b[1;36m4.0063e-03\u001b[0m,  \u001b[1;36m2.4540e-03\u001b[0m, \u001b[1;36m-9.4763e-02\u001b[0m,  \u001b[1;36m5.6738e-02\u001b[0m, \u001b[1;36m-8.9051e-03\u001b[0m,\n",
       "        \u001b[1;36m-1.5367e-02\u001b[0m,  \u001b[1;36m3.7929e-02\u001b[0m, \u001b[1;36m-3.1128e-02\u001b[0m, \u001b[1;36m-5.5165e-02\u001b[0m, \u001b[1;36m-2.0374e-01\u001b[0m,\n",
       "         \u001b[1;36m1.0024e-01\u001b[0m,  \u001b[1;36m8.9946e-02\u001b[0m, \u001b[1;36m-8.1773e-03\u001b[0m, \u001b[1;36m-4.6732e-02\u001b[0m,  \u001b[1;36m5.4862e-02\u001b[0m,\n",
       "        \u001b[1;36m-4.9572e-02\u001b[0m,  \u001b[1;36m1.6269e-02\u001b[0m,  \u001b[1;36m5.5046e-02\u001b[0m, \u001b[1;36m-4.6725e-03\u001b[0m, \u001b[1;36m-2.6217e-02\u001b[0m,\n",
       "        \u001b[1;36m-2.7294e-03\u001b[0m,  \u001b[1;36m1.0804e-01\u001b[0m,  \u001b[1;36m4.4482e-02\u001b[0m, \u001b[1;36m-3.9526e-02\u001b[0m,  \u001b[1;36m4.1552e-02\u001b[0m,\n",
       "         \u001b[1;36m4.2067e-02\u001b[0m,  \u001b[1;36m6.9128e-02\u001b[0m, \u001b[1;36m-7.0026e-02\u001b[0m, \u001b[1;36m-4.0075e-02\u001b[0m, \u001b[1;36m-4.2712e-04\u001b[0m,\n",
       "         \u001b[1;36m2.8174e-02\u001b[0m,  \u001b[1;36m4.9838e-02\u001b[0m, \u001b[1;36m-9.1041e-02\u001b[0m, \u001b[1;36m-6.0918e-02\u001b[0m, \u001b[1;36m-5.0477e-02\u001b[0m,\n",
       "        \u001b[1;36m-8.3952e-02\u001b[0m, \u001b[1;36m-1.4577e-03\u001b[0m, \u001b[1;36m-6.7692e-03\u001b[0m,  \u001b[1;36m4.1205e-02\u001b[0m,  \u001b[1;36m3.3702e-02\u001b[0m,\n",
       "         \u001b[1;36m2.6134e-02\u001b[0m, \u001b[1;36m-9.3400e-02\u001b[0m, \u001b[1;36m-1.7563e-02\u001b[0m, \u001b[1;36m-5.4995e-02\u001b[0m,  \u001b[1;36m1.2440e-01\u001b[0m,\n",
       "        \u001b[1;36m-8.1775e-03\u001b[0m, \u001b[1;36m-1.0323e-01\u001b[0m,  \u001b[1;36m2.4818e-02\u001b[0m, \u001b[1;36m-6.4749e-02\u001b[0m, \u001b[1;36m-3.5071e-02\u001b[0m,\n",
       "         \u001b[1;36m3.1912e-04\u001b[0m, \u001b[1;36m-4.1279e-02\u001b[0m, \u001b[1;36m-1.4959e-01\u001b[0m, \u001b[1;36m-2.2634e-02\u001b[0m,  \u001b[1;36m5.8337e-02\u001b[0m,\n",
       "         \u001b[1;36m7.4144e-03\u001b[0m,  \u001b[1;36m6.4620e-03\u001b[0m,  \u001b[1;36m3.8110e-02\u001b[0m,  \u001b[1;36m4.3539e-02\u001b[0m, \u001b[1;36m-5.4276e-02\u001b[0m,\n",
       "         \u001b[1;36m8.7545e-02\u001b[0m, \u001b[1;36m-1.4871e-02\u001b[0m,  \u001b[1;36m9.6427e-02\u001b[0m, \u001b[1;36m-2.2182e-02\u001b[0m,  \u001b[1;36m4.9424e-02\u001b[0m,\n",
       "        \u001b[1;36m-1.6440e-02\u001b[0m, \u001b[1;36m-4.0981e-02\u001b[0m,  \u001b[1;36m2.6056e-02\u001b[0m, \u001b[1;36m-5.1585e-02\u001b[0m, \u001b[1;36m-3.1748e-02\u001b[0m,\n",
       "         \u001b[1;36m1.0050e-02\u001b[0m, \u001b[1;36m-1.0158e-01\u001b[0m, \u001b[1;36m-7.4725e-02\u001b[0m,  \u001b[1;36m5.3766e-02\u001b[0m,  \u001b[1;36m5.9533e-02\u001b[0m,\n",
       "        \u001b[1;36m-8.9224e-03\u001b[0m,  \u001b[1;36m3.0551e-02\u001b[0m,  \u001b[1;36m8.3009e-02\u001b[0m,  \u001b[1;36m2.2091e-02\u001b[0m, \u001b[1;36m-1.6830e-02\u001b[0m,\n",
       "        \u001b[1;36m-1.2798e-02\u001b[0m, \u001b[1;36m-5.4892e-02\u001b[0m,  \u001b[1;36m2.0980e-02\u001b[0m, \u001b[1;36m-1.0143e-01\u001b[0m,  \u001b[1;36m8.2567e-03\u001b[0m,\n",
       "         \u001b[1;36m5.7449e-03\u001b[0m,  \u001b[1;36m1.2578e-02\u001b[0m, \u001b[1;36m-4.7935e-02\u001b[0m, \u001b[1;36m-2.4858e-02\u001b[0m,  \u001b[1;36m3.1339e-02\u001b[0m,\n",
       "         \u001b[1;36m9.7304e-02\u001b[0m, \u001b[1;36m-2.1196e-02\u001b[0m, \u001b[1;36m-2.9524e-02\u001b[0m, \u001b[1;36m-3.0847e-02\u001b[0m, \u001b[1;36m-1.8560e-08\u001b[0m,\n",
       "        \u001b[1;36m-5.2432e-02\u001b[0m, \u001b[1;36m-7.1539e-03\u001b[0m,  \u001b[1;36m2.7848e-03\u001b[0m,  \u001b[1;36m1.3505e-02\u001b[0m, \u001b[1;36m-1.8513e-03\u001b[0m,\n",
       "         \u001b[1;36m3.4044e-02\u001b[0m, \u001b[1;36m-5.6208e-02\u001b[0m, \u001b[1;36m-6.3033e-02\u001b[0m,  \u001b[1;36m2.5686e-02\u001b[0m,  \u001b[1;36m6.0419e-02\u001b[0m,\n",
       "        \u001b[1;36m-2.5544e-02\u001b[0m, \u001b[1;36m-5.4425e-02\u001b[0m, \u001b[1;36m-6.2813e-03\u001b[0m,  \u001b[1;36m2.0601e-02\u001b[0m, \u001b[1;36m-1.3528e-02\u001b[0m,\n",
       "         \u001b[1;36m2.4703e-02\u001b[0m,  \u001b[1;36m7.7846e-02\u001b[0m,  \u001b[1;36m2.3287e-02\u001b[0m, \u001b[1;36m-3.3436e-02\u001b[0m,  \u001b[1;36m6.4842e-02\u001b[0m,\n",
       "        \u001b[1;36m-4.4627e-02\u001b[0m,  \u001b[1;36m1.1172e-01\u001b[0m,  \u001b[1;36m1.5232e-03\u001b[0m, \u001b[1;36m-5.2231e-02\u001b[0m, \u001b[1;36m-4.1692e-02\u001b[0m,\n",
       "         \u001b[1;36m6.6046e-02\u001b[0m,  \u001b[1;36m1.7801e-02\u001b[0m,  \u001b[1;36m2.7188e-02\u001b[0m, \u001b[1;36m-1.3282e-02\u001b[0m,  \u001b[1;36m6.9399e-02\u001b[0m,\n",
       "        \u001b[1;36m-2.3109e-02\u001b[0m, \u001b[1;36m-1.5641e-02\u001b[0m, \u001b[1;36m-4.6476e-02\u001b[0m, \u001b[1;36m-3.3069e-04\u001b[0m,  \u001b[1;36m7.5727e-02\u001b[0m,\n",
       "        \u001b[1;36m-1.3049e-02\u001b[0m,  \u001b[1;36m8.1173e-02\u001b[0m, \u001b[1;36m-6.3903e-02\u001b[0m,  \u001b[1;36m4.9146e-02\u001b[0m,  \u001b[1;36m3.6656e-02\u001b[0m,\n",
       "        \u001b[1;36m-3.1578e-02\u001b[0m,  \u001b[1;36m7.6392e-02\u001b[0m,  \u001b[1;36m7.1512e-03\u001b[0m,  \u001b[1;36m9.1036e-04\u001b[0m,  \u001b[1;36m3.4462e-02\u001b[0m,\n",
       "         \u001b[1;36m7.1568e-02\u001b[0m,  \u001b[1;36m1.2789e-02\u001b[0m,  \u001b[1;36m1.5305e-02\u001b[0m,  \u001b[1;36m1.0297e-02\u001b[0m,  \u001b[1;36m3.0290e-02\u001b[0m,\n",
       "        \u001b[1;36m-4.6647e-02\u001b[0m, \u001b[1;36m-1.1118e-02\u001b[0m, \u001b[1;36m-1.6685e-03\u001b[0m, \u001b[1;36m-7.5788e-02\u001b[0m,  \u001b[1;36m2.5742e-02\u001b[0m,\n",
       "        \u001b[1;36m-3.1140e-02\u001b[0m, \u001b[1;36m-2.8856e-02\u001b[0m, \u001b[1;36m-6.4451e-02\u001b[0m,  \u001b[1;36m4.0715e-02\u001b[0m, \u001b[1;36m-3.2946e-02\u001b[0m,\n",
       "        \u001b[1;36m-1.1134e-02\u001b[0m,  \u001b[1;36m5.0589e-02\u001b[0m, \u001b[1;36m-1.3195e-02\u001b[0m,  \u001b[1;36m3.2110e-02\u001b[0m\u001b[1m]\u001b[0m, \u001b[33mdevice\u001b[0m=\u001b[32m'cuda:0'\u001b[0m\u001b[1m)\u001b[0m\n",
       "\u001b[1m}\u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "resp = model.encode(text, output_value=None)\n",
    "rich.print(resp)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e39f581e-e0c3-47e8-a5af-c3a94d5214f8",
   "metadata": {},
   "source": [
    "number of tokens if based on whitespace"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "id": "1eb3e125-aa23-4432-b5b2-57711981d944",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-26T12:56:09.113228Z",
     "iopub.status.busy": "2023-04-26T12:56:09.112521Z",
     "iopub.status.idle": "2023-04-26T12:56:09.118130Z",
     "shell.execute_reply": "2023-04-26T12:56:09.117253Z",
     "shell.execute_reply.started": "2023-04-26T12:56:09.113190Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "59"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(text)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "155fc68d-3959-485a-8902-a2ddb194b6d5",
   "metadata": {},
   "source": [
    "number of tokens according to the model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "id": "6c52b171-cff8-4673-a66b-525980b9f3b5",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-26T12:56:09.920728Z",
     "iopub.status.busy": "2023-04-26T12:56:09.920029Z",
     "iopub.status.idle": "2023-04-26T12:56:09.925512Z",
     "shell.execute_reply": "2023-04-26T12:56:09.924752Z",
     "shell.execute_reply.started": "2023-04-26T12:56:09.920695Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([18])"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "resp['input_ids'].shape "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e86382cb-3a72-4fd4-a4c1-b449ded38deb",
   "metadata": {},
   "source": [
    "each token has an embedding"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "id": "66101a29-cf6b-460a-befb-8f1093555303",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-26T12:56:10.720943Z",
     "iopub.status.busy": "2023-04-26T12:56:10.720084Z",
     "iopub.status.idle": "2023-04-26T12:56:10.725559Z",
     "shell.execute_reply": "2023-04-26T12:56:10.724729Z",
     "shell.execute_reply.started": "2023-04-26T12:56:10.720911Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([18, 384])"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "resp['token_embeddings'].shape\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c009dd6e-cd1d-4376-8194-b97eb81bef8f",
   "metadata": {},
   "source": [
    "All the token are pooled to generate an embedding"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "id": "efa0ae73-1609-4333-ba94-4d5163429e18",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-26T12:56:11.725613Z",
     "iopub.status.busy": "2023-04-26T12:56:11.724716Z",
     "iopub.status.idle": "2023-04-26T12:56:11.730998Z",
     "shell.execute_reply": "2023-04-26T12:56:11.729727Z",
     "shell.execute_reply.started": "2023-04-26T12:56:11.725577Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "torch.Size([384])"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "resp['sentence_embedding'].shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "472d06b1-c74f-40e3-a1fa-54c534575f6e",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "fd6dad3e-2af2-4bb1-89d0-759b78f0215f",
   "metadata": {},
   "source": [
    "### Explore the tokenizer more"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "id": "9cdfef33-ccbd-4a44-87bd-8d337ba7cb10",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-26T12:56:12.804516Z",
     "iopub.status.busy": "2023-04-26T12:56:12.804123Z",
     "iopub.status.idle": "2023-04-26T12:56:12.959413Z",
     "shell.execute_reply": "2023-04-26T12:56:12.958339Z",
     "shell.execute_reply.started": "2023-04-26T12:56:12.804483Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "tokenizer = AutoTokenizer.from_pretrained(model_name)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c7a3d63-1310-4bd4-8a4c-efce530d8ab5",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-26T12:55:58.933986Z",
     "iopub.status.busy": "2023-04-26T12:55:58.933147Z",
     "iopub.status.idle": "2023-04-26T12:55:58.938744Z",
     "shell.execute_reply": "2023-04-26T12:55:58.937599Z",
     "shell.execute_reply.started": "2023-04-26T12:55:58.933951Z"
    }
   },
   "source": [
    "the number of unique words in the tokenizer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "id": "9895b643-1441-49f5-8cdf-710abce57602",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-26T12:56:13.859218Z",
     "iopub.status.busy": "2023-04-26T12:56:13.858227Z",
     "iopub.status.idle": "2023-04-26T12:56:13.864864Z",
     "shell.execute_reply": "2023-04-26T12:56:13.863874Z",
     "shell.execute_reply.started": "2023-04-26T12:56:13.859179Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "30522"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tokenizer.vocab_size"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "id": "e93b4f92-dd9d-4b60-a526-bc1370c80fa7",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-26T12:56:49.928726Z",
     "iopub.status.busy": "2023-04-26T12:56:49.928076Z",
     "iopub.status.idle": "2023-04-26T12:56:49.934073Z",
     "shell.execute_reply": "2023-04-26T12:56:49.933364Z",
     "shell.execute_reply.started": "2023-04-26T12:56:49.928694Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'input_ids': [101, 1037, 2158, 1998, 2010, 9004, 3899, 2566, 28414, 2377, 4667, 2094, 4608, 1999, 1996, 3509, 1012, 102], 'token_type_ids': [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], 'attention_mask': [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]}"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "resp = tokenizer(text)\n",
    "resp"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "596ab830-ef70-4af5-88cd-cd15c6a42dfe",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "id": "1d976f1f-d467-4a6e-a8fb-5fa4ab3d29c0",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-26T12:56:20.500712Z",
     "iopub.status.busy": "2023-04-26T12:56:20.499861Z",
     "iopub.status.idle": "2023-04-26T12:56:20.507809Z",
     "shell.execute_reply": "2023-04-26T12:56:20.506907Z",
     "shell.execute_reply.started": "2023-04-26T12:56:20.500680Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\">[</span>\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'[CLS]'</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'a'</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'man'</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'and'</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'his'</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'pet'</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'dog'</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'per'</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'##rito'</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'play'</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'##ding'</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'##d'</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'catch'</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'in'</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'the'</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'beach'</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'.'</span>,\n",
       "    <span style=\"color: #008000; text-decoration-color: #008000\">'[SEP]'</span>\n",
       "<span style=\"font-weight: bold\">]</span>\n",
       "</pre>\n"
      ],
      "text/plain": [
       "\u001b[1m[\u001b[0m\n",
       "    \u001b[32m'\u001b[0m\u001b[32m[\u001b[0m\u001b[32mCLS\u001b[0m\u001b[32m]\u001b[0m\u001b[32m'\u001b[0m,\n",
       "    \u001b[32m'a'\u001b[0m,\n",
       "    \u001b[32m'man'\u001b[0m,\n",
       "    \u001b[32m'and'\u001b[0m,\n",
       "    \u001b[32m'his'\u001b[0m,\n",
       "    \u001b[32m'pet'\u001b[0m,\n",
       "    \u001b[32m'dog'\u001b[0m,\n",
       "    \u001b[32m'per'\u001b[0m,\n",
       "    \u001b[32m'##rito'\u001b[0m,\n",
       "    \u001b[32m'play'\u001b[0m,\n",
       "    \u001b[32m'##ding'\u001b[0m,\n",
       "    \u001b[32m'##d'\u001b[0m,\n",
       "    \u001b[32m'catch'\u001b[0m,\n",
       "    \u001b[32m'in'\u001b[0m,\n",
       "    \u001b[32m'the'\u001b[0m,\n",
       "    \u001b[32m'beach'\u001b[0m,\n",
       "    \u001b[32m'.'\u001b[0m,\n",
       "    \u001b[32m'\u001b[0m\u001b[32m[\u001b[0m\u001b[32mSEP\u001b[0m\u001b[32m]\u001b[0m\u001b[32m'\u001b[0m\n",
       "\u001b[1m]\u001b[0m\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "tokens = tokenizer.tokenize(text,add_special_tokens=True)\n",
    "rich.print ( tokens)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "id": "cdc8a452-cc22-4624-84fc-472cd880cb30",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-26T12:57:06.192962Z",
     "iopub.status.busy": "2023-04-26T12:57:06.192139Z",
     "iopub.status.idle": "2023-04-26T12:57:06.198717Z",
     "shell.execute_reply": "2023-04-26T12:57:06.197600Z",
     "shell.execute_reply.started": "2023-04-26T12:57:06.192930Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'[CLS] a man and his pet dog perrito playdingd catch in the beach. [SEP]'"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "decoded_string = tokenizer.decode(resp['input_ids'])\n",
    "decoded_string"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0dd2c4fb-08dc-4e71-9773-afedf84182cb",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3a83c00d-016b-4bcb-a85b-83ee2dea9b62",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8baa5a71-6f11-45eb-b60c-ec34bb8ec1ec",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "394d6bce-2e0c-4c5d-9638-ff92bbbdb421",
   "metadata": {},
   "source": [
    "What happens if the input query is longer than the query"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ac529a8a-02a9-442d-85c0-b46bce453fe9",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-25T23:04:51.651243Z",
     "iopub.status.busy": "2023-04-25T23:04:51.650538Z",
     "iopub.status.idle": "2023-04-25T23:04:51.656468Z",
     "shell.execute_reply": "2023-04-25T23:04:51.655709Z",
     "shell.execute_reply.started": "2023-04-25T23:04:51.651209Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "# movie text of Puss in Boots https://en.wikipedia.org/wiki/Puss_in_Boots:_The_Last_Wish\n",
    "text = f\"\"\"\n",
    "\n",
    "Puss in Boots: The Last Wish is a 2022 American computer-animated adventure film produced by DreamWorks Animation and distributed by Universal Pictures. \n",
    "\n",
    "The sequel to the spin-off film Puss in Boots (2011) and the sixth installment in the Shrek franchise, the film was directed by Joel Crawford and co-directed by Januel Mercado. \n",
    "\n",
    "Based on the character from Shrek 2 (2004) and inspired by the eponymous fairy tale, the film's screenplay was written by Paul Fisher and Tommy Swerdlow, with a story by Swerdlow and Tom Wheeler (the latter of whom wrote the 2011 film). \n",
    "\n",
    "The voice cast of Puss in Boots: The Last Wish includes Antonio Banderas and Salma Hayek Pinault reprising their respective roles as the titular character and Kitty Softpaws, and are joined by Harvey Guillén, Florence Pugh, Olivia Colman, Ray Winstone, Samson Kayo, John Mulaney, Wagner Moura, Da'Vine Joy Randolph, and Anthony Mendez, who voice new characters introduced in the film. \n",
    "\n",
    "In the film, after the events of Shrek Forever After (2010), Puss in Boots teams up with Kitty and Perrito (Guillén) to find the mystical Last Wish for the Wishing Star to restore the first eight of his nine lives, before Goldilocks and her Three Bears Crime Family (Pugh, Winstone, Colman, and Kayo), and \"Big\" Jack Horner (Mulaney) do, while attempting to avoid a mysterious hooded wolf (Moura).\n",
    "\n",
    "\n",
    "Plans for a sequel to Puss in Boots began in November 2012, when executive producer Guillermo del Toro shared plans to take the titular character on an adventure to a \"very exotic locale\". Work on a sequel began in April 2014, according to Banderas. After being stuck in development hell, the project was revived in November 2018, with Illumination founder and CEO Chris Meledandri confirmed to be an executive producer. It was announced that the film would be helmed by Bob Persichetti, the head of story of the first film and one of the three directors of Sony Pictures Animation's Spider-Man: Into the Spider-Verse (2018), in February 2019. Crawford was later announced as the new director in March 2021, along with Mercado. The story draws inspiration from Spaghetti Western films, with The Good, the Bad and the Ugly (1966) being cited as a particular influence. As with DreamWorks' previous film The Bad Guys (2022), the film has a stylized animation style, inspired by Spider-Man: Into the Spider-Verse. With new technology, the team was able to give the film a painterly style to resemble a fairy-tale story, rather than utilizing the realistic visual style of every previous installment of the Shrek franchise.\n",
    "\n",
    "Puss in Boots: The Last Wish premiered at Lincoln Center in New York City on December 13, 2022, and was theatrically released in the United States on December 21, 2022, after being delayed due to restructuring at DreamWorks. The film received positive reviews from critics and was a commercial success, grossing $483 million worldwide on a production budget of $90–110 million, becoming the tenth-highest-grossing film of 2022. At the 95th Academy Awards it was nominated for Best Animated Feature, as well as receiving nominations at the Golden Globes, Critics' Choice Awards, and British Academy Film Awards.\n",
    "\"\"\"\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "d7a367e4-4b82-45f8-ab3b-43e921f74e79",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-26T12:54:36.246478Z",
     "iopub.status.busy": "2023-04-26T12:54:36.245585Z",
     "iopub.status.idle": "2023-04-26T12:54:36.251748Z",
     "shell.execute_reply": "2023-04-26T12:54:36.251036Z",
     "shell.execute_reply.started": "2023-04-26T12:54:36.246444Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "tokens = tokenizer.tokenize(text) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "9d516fd3-7224-4290-b6cc-c04ac631300c",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-26T12:54:36.916777Z",
     "iopub.status.busy": "2023-04-26T12:54:36.916427Z",
     "iopub.status.idle": "2023-04-26T12:54:36.922267Z",
     "shell.execute_reply": "2023-04-26T12:54:36.921363Z",
     "shell.execute_reply.started": "2023-04-26T12:54:36.916749Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "710"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(tokens)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "33509f06-0aaa-4670-b769-7dfd895be225",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-26T12:54:37.368115Z",
     "iopub.status.busy": "2023-04-26T12:54:37.367760Z",
     "iopub.status.idle": "2023-04-26T12:54:37.373877Z",
     "shell.execute_reply": "2023-04-26T12:54:37.372969Z",
     "shell.execute_reply.started": "2023-04-26T12:54:37.368087Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(3194, 512)"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(text),  tokenizer.model_max_length"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "1a02aecd-1258-4449-a124-c279d5a7c78e",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-26T12:54:38.172339Z",
     "iopub.status.busy": "2023-04-26T12:54:38.171560Z",
     "iopub.status.idle": "2023-04-26T12:54:38.176655Z",
     "shell.execute_reply": "2023-04-26T12:54:38.175690Z",
     "shell.execute_reply.started": "2023-04-26T12:54:38.172309Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "pu ##ss in boots : the last wish is a 202 ##2 american computer - animated adventure film produced by dream ##works animation and distributed by universal pictures . the sequel to the spin - off film pu ##ss in boots ( 2011 ) and the sixth installment in the sh ##rek franchise , the film was directed by joel crawford and co - directed by jan ##uel mer ##ca ##do . based on the character from sh ##rek 2 ( 2004 ) and inspired by the eponymous fairy tale , the film ' s screenplay was written by paul fisher and tommy sw ##er ##dlow , with a story by sw ##er ##dlow and tom wheeler ( the latter of whom wrote the 2011 film ) . the voice cast of pu ##ss in boots : the last wish includes antonio band ##eras and sal ##ma hay ##ek pin ##ault rep ##ris ##ing their respective roles as the titular character and kitty soft ##pa ##ws , and are joined by harvey gui ##llen , florence pu ##gh , olivia col ##man , ray winston ##e , samson kay ##o , john mu ##lane ##y , wagner mo ##ura , da ' vine joy randolph , and anthony mendez , who voice new characters introduced in the film . in the film , after the events of sh ##rek forever after ( 2010 ) , pu ##ss in boots teams up with kitty and per ##rito ( gui ##llen ) to find the mystical last wish for the wishing star to restore the first eight of his nine lives , before gold ##ilo ##cks and her three bears crime family ( pu ##gh , winston ##e , col ##man , and kay ##o ) , and \" big \" jack horne ##r ( mu ##lane ##y ) do , while attempting to avoid a mysterious hooded wolf ( mo ##ura ) . plans for a sequel to pu ##ss in boots began in november 2012 , when executive producer guillermo del toro shared plans to take the titular character on an adventure to a \" very exotic local ##e \" . work on a sequel began in april 2014 , according to band ##eras . after being stuck in development hell , the project was revived in november 2018 , with illumination founder and ceo chris mel ##eda ##nd ##ri confirmed to be an executive producer . it was announced that the film would be helm ##ed by bob per ##sic ##hett ##i , the head of story of the first film and one of the three directors of sony pictures animation ' s spider - man : into the spider - verse ( 2018 ) , in february 2019 . crawford was later announced as the new director in march 2021 , along with mer ##ca ##do . the story draws inspiration from spaghetti western films , with the good , the bad and the ugly ( 1966 ) being cited as a particular influence . as with dream ##works ' previous film the bad guys ( 202 ##2 ) , the film has a stylized animation style , inspired by spider - man : into the spider - verse . with new technology , the team was able to give the film a painter ##ly style to resemble a fairy - tale story , rather than utilizing the realistic visual style of every previous installment of the sh ##rek franchise . pu ##ss in boots : the last wish premiered at lincoln center in new york city on december 13 , 202 ##2 , and was theatrical ##ly released in the united states on december 21 , 202 ##2 , after being delayed due to restructuring at dream ##works . the film received positive reviews from critics and was a commercial success , grossing $ 48 ##3 million worldwide on a production budget of $ 90 – 110 million , becoming the tenth - highest - grossing film of 202 ##2 . at the 95 ##th academy awards it was nominated for best animated feature , as well as receiving nominations at the golden globe ##s , critics ' choice awards , and british academy film awards .\n"
     ]
    }
   ],
   "source": [
    "print (\" \".join( tokens ))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "efab9178-bbd9-4391-988a-b29ba95eccaa",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "6d549eaa-2828-473e-952d-b60f9817ae7c",
   "metadata": {},
   "source": [
    "tokens after `512` position are skipped by the model.   \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a7e32a06-6c59-40f8-b572-8ec581e5dafb",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "30f4c12e-fc54-4d02-9f67-4a79de6c1f73",
   "metadata": {},
   "source": [
    "compute the embeddings of the entire corpus"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eff3908f-1fbe-4dd5-9e67-bc8f34c8a909",
   "metadata": {},
   "source": [
    "# Embeddings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "id": "3435fe9a-03e3-40f0-9599-de724b61fe94",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-25T23:08:35.898478Z",
     "iopub.status.busy": "2023-04-25T23:08:35.897834Z",
     "iopub.status.idle": "2023-04-25T23:08:35.949598Z",
     "shell.execute_reply": "2023-04-25T23:08:35.948731Z",
     "shell.execute_reply.started": "2023-04-25T23:08:35.898441Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\u001b[0;31mSignature:\u001b[0m\n",
       "\u001b[0mmodel\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mencode\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0msentences\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mUnion\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mstr\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mList\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mstr\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mbatch_size\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mint\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m32\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mshow_progress_bar\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mbool\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0moutput_value\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mstr\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m'sentence_embedding'\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mconvert_to_numpy\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mbool\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mTrue\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mconvert_to_tensor\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mbool\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mdevice\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mstr\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m    \u001b[0mnormalize_embeddings\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0mbool\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;32mFalse\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\n",
       "\u001b[0;34m\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m->\u001b[0m \u001b[0mUnion\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mList\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mtorch\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mTensor\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnumpy\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mndarray\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mtorch\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mTensor\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
       "\u001b[0;31mDocstring:\u001b[0m\n",
       "Computes sentence embeddings\n",
       "\n",
       ":param sentences: the sentences to embed\n",
       ":param batch_size: the batch size used for the computation\n",
       ":param show_progress_bar: Output a progress bar when encode sentences\n",
       ":param output_value:  Default sentence_embedding, to get sentence embeddings. Can be set to token_embeddings to get wordpiece token embeddings. Set to None, to get all output values\n",
       ":param convert_to_numpy: If true, the output is a list of numpy vectors. Else, it is a list of pytorch tensors.\n",
       ":param convert_to_tensor: If true, you get one large tensor as return. Overwrites any setting from convert_to_numpy\n",
       ":param device: Which torch.device to use for the computation\n",
       ":param normalize_embeddings: If set to true, returned vectors will have length 1. In that case, the faster dot-product (util.dot_score) instead of cosine similarity can be used.\n",
       "\n",
       ":return:\n",
       "   By default, a list of tensors is returned. If convert_to_tensor, a stacked tensor is returned. If convert_to_numpy, a numpy matrix is returned.\n",
       "\u001b[0;31mFile:\u001b[0m      /opt/conda/lib/python3.7/site-packages/sentence_transformers/SentenceTransformer.py\n",
       "\u001b[0;31mType:\u001b[0m      method\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "?model.encode"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "id": "e72bde4e-9a32-4800-b434-1afac6306bb3",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-25T23:09:15.868261Z",
     "iopub.status.busy": "2023-04-25T23:09:15.867546Z",
     "iopub.status.idle": "2023-04-25T23:09:22.432792Z",
     "shell.execute_reply": "2023-04-25T23:09:22.431949Z",
     "shell.execute_reply.started": "2023-04-25T23:09:15.868226Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "corpus = model.encode(dset['description_final'], normalize_embeddings=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "id": "0d3857de-e224-4233-b805-e23dc5bda7a8",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-25T23:13:29.093898Z",
     "iopub.status.busy": "2023-04-25T23:13:29.093036Z",
     "iopub.status.idle": "2023-04-25T23:13:29.101355Z",
     "shell.execute_reply": "2023-04-25T23:13:29.100523Z",
     "shell.execute_reply.started": "2023-04-25T23:13:29.093864Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "def find_results(query:str , k =5, explain=False,image_path_local_dir = \"../data/raw/images\"):\n",
    "    \n",
    "    # get embeddings of a query\n",
    "    query_features = model.encode(query, normalize_embeddings=True)\n",
    "    \n",
    "    # compute cosine scores of the scores and all the other documents in the corpus\n",
    "    doc_scores = query_features @ corpus.T\n",
    "    \n",
    "    # get index of top items\n",
    "    top_items_idx = doc_scores.argsort()[-k:][::-1]\n",
    "\n",
    "\n",
    "    tokens = tokenizer.tokenize(query) \n",
    "    degug_info = {\n",
    "         \"query_original\":  query \n",
    "         , \"query_processed\" : query_features\n",
    "         , \"tokens\": tokens\n",
    "        , \"doc_scores\":  doc_scores \n",
    "        , \"top_items\":  top_items_idx \n",
    "    }\n",
    "    \n",
    "    \n",
    "\n",
    "    display(HTML(f\"<h4>Query: {query} </h4>\"))\n",
    "    \n",
    "    if explain:\n",
    "        rich.print (degug_info )\n",
    "        \n",
    "        \n",
    "    \n",
    "    images = []\n",
    "    labels = []\n",
    "    \n",
    "    \n",
    "  \n",
    "    print(tokens)\n",
    "    # Iterate over the top k results\n",
    "    \n",
    "    for idx, photo_data in enumerate( dset.select(top_items_idx)):\n",
    "\n",
    "        \n",
    "        doc_idx = top_items_idx[idx]\n",
    "        \n",
    "        photo_id = photo_data['photo_id']\n",
    "        \n",
    "        image_path_local = f\"{image_path_local_dir}/{photo_id}.jpg\"\n",
    "\n",
    "        if os.path.exists(image_path_local):\n",
    "            image_path = image_path_local\n",
    "        else:\n",
    "            image_path = photo_data[\"photo_image_url\"]\n",
    "        \n",
    "        images.append(image_path)       \n",
    "        score = \"{:.5f}\".format(doc_scores[doc_idx])\n",
    "\n",
    "        labels.append (f\"\"\"\n",
    "                     Photo title: {photo_data[\"description_final\"]}   <br/>\n",
    "                     Distance: {score}\n",
    "            \n",
    "                     \"\"\")\n",
    "        \n",
    "    ipyplot.plot_images(images=images, labels=labels, img_width=200)\n",
    "\n",
    "        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "41ac0972-1a60-426e-bd19-a7dead068b48",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "e7d1b1af-a36d-497f-92c4-abcce9afb7f6",
   "metadata": {},
   "source": [
    "# More Examples"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "id": "9c4a6fd0-9f4f-455d-86eb-02a8140387a1",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-25T23:13:29.924461Z",
     "iopub.status.busy": "2023-04-25T23:13:29.923611Z",
     "iopub.status.idle": "2023-04-25T23:13:30.013705Z",
     "shell.execute_reply": "2023-04-25T23:13:30.012407Z",
     "shell.execute_reply.started": "2023-04-25T23:13:29.924429Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<h4>Query: Two dogs playing in the snow </h4>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['two', 'dogs', 'playing', 'in', 'the', 'snow']\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <style>\n",
       "        #ipyplot-html-viewer-toggle-gN7PN56dPNsdR5DuwKMhgT {\n",
       "            position: absolute;\n",
       "            top: -9999px;\n",
       "            left: -9999px;\n",
       "            visibility: hidden;\n",
       "        }\n",
       "\n",
       "        #ipyplot-html-viewer-label-gN7PN56dPNsdR5DuwKMhgT { \n",
       "            position: relative;\n",
       "            display: inline-block;\n",
       "            cursor: pointer;\n",
       "            color: blue;\n",
       "            text-decoration: underline;\n",
       "        }\n",
       "\n",
       "        #ipyplot-html-viewer-textarea-gN7PN56dPNsdR5DuwKMhgT {\n",
       "            background: lightgrey;\n",
       "            width: 100%;\n",
       "            height: 0px;\n",
       "            display: none;\n",
       "        }\n",
       "\n",
       "        #ipyplot-html-viewer-toggle-gN7PN56dPNsdR5DuwKMhgT:checked ~ #ipyplot-html-viewer-textarea-gN7PN56dPNsdR5DuwKMhgT {\n",
       "            height: 200px;\n",
       "            display: block;\n",
       "        }\n",
       "\n",
       "        #ipyplot-html-viewer-toggle-gN7PN56dPNsdR5DuwKMhgT:checked + #ipyplot-html-viewer-label-gN7PN56dPNsdR5DuwKMhgT:after {\n",
       "            content: \"hide html\";\n",
       "            position: absolute;\n",
       "            top: 0;\n",
       "            left: 0;\n",
       "            right: 0;\n",
       "            bottom: 0;\n",
       "            background: white;\n",
       "            cursor: pointer;\n",
       "            color: blue;\n",
       "            text-decoration: underline;\n",
       "        }\n",
       "    </style>\n",
       "    <div>\n",
       "        <input type=\"checkbox\" id=\"ipyplot-html-viewer-toggle-gN7PN56dPNsdR5DuwKMhgT\">\n",
       "        <label id=\"ipyplot-html-viewer-label-gN7PN56dPNsdR5DuwKMhgT\" for=\"ipyplot-html-viewer-toggle-gN7PN56dPNsdR5DuwKMhgT\">show html</label>\n",
       "        <textarea id=\"ipyplot-html-viewer-textarea-gN7PN56dPNsdR5DuwKMhgT\" readonly>\n",
       "            \n",
       "        <style>\n",
       "        #ipyplot-imgs-container-div-ZmBbGKRVkcBs8r4hmraygy {\n",
       "            width: 100%;\n",
       "            height: 100%;\n",
       "            margin: 0%;\n",
       "            overflow: auto;\n",
       "            position: relative;\n",
       "            overflow-y: scroll;\n",
       "        }\n",
       "\n",
       "        div.ipyplot-placeholder-div-ZmBbGKRVkcBs8r4hmraygy {\n",
       "            width: 200px;\n",
       "            display: inline-block;\n",
       "            margin: 3px;\n",
       "            position: relative;\n",
       "        }\n",
       "\n",
       "        div.ipyplot-content-div-ZmBbGKRVkcBs8r4hmraygy {\n",
       "            width: 200px;\n",
       "            background: white;\n",
       "            display: inline-block;\n",
       "            vertical-align: top;\n",
       "            text-align: center;\n",
       "            position: relative;\n",
       "            border: 2px solid #ddd;\n",
       "            top: 0;\n",
       "            left: 0;\n",
       "        }\n",
       "\n",
       "        div.ipyplot-content-div-ZmBbGKRVkcBs8r4hmraygy span.ipyplot-img-close {\n",
       "            display: none;\n",
       "        }\n",
       "\n",
       "        div.ipyplot-content-div-ZmBbGKRVkcBs8r4hmraygy span {\n",
       "            width: 100%;\n",
       "            height: 100%;\n",
       "            position: absolute;\n",
       "            top: 0;\n",
       "            left: 0;\n",
       "        }\n",
       "\n",
       "        div.ipyplot-content-div-ZmBbGKRVkcBs8r4hmraygy img {\n",
       "            width: 200px;\n",
       "        }\n",
       "\n",
       "        div.ipyplot-content-div-ZmBbGKRVkcBs8r4hmraygy span.ipyplot-img-close:hover {\n",
       "            cursor: zoom-out;\n",
       "        }\n",
       "        div.ipyplot-content-div-ZmBbGKRVkcBs8r4hmraygy span.ipyplot-img-expand:hover {\n",
       "            cursor: zoom-in;\n",
       "        }\n",
       "\n",
       "        div[id^=ipyplot-content-div-ZmBbGKRVkcBs8r4hmraygy]:target {\n",
       "            transform: scale(2.5);\n",
       "            transform-origin: left top;\n",
       "            z-index: 5000;\n",
       "            top: 0;\n",
       "            left: 0;\n",
       "            position: absolute;\n",
       "        }\n",
       "\n",
       "        div[id^=ipyplot-content-div-ZmBbGKRVkcBs8r4hmraygy]:target span.ipyplot-img-close {\n",
       "            display: block;\n",
       "        }\n",
       "\n",
       "        div[id^=ipyplot-content-div-ZmBbGKRVkcBs8r4hmraygy]:target span.ipyplot-img-expand {\n",
       "            display: none;\n",
       "        }\n",
       "        </style>\n",
       "    <div id=\"ipyplot-imgs-container-div-ZmBbGKRVkcBs8r4hmraygy\">\n",
       "    <div class=\"ipyplot-placeholder-div-ZmBbGKRVkcBs8r4hmraygy\">\n",
       "        <div id=\"ipyplot-content-div-ZmBbGKRVkcBs8r4hmraygy-X4LCbG2mxoxuQtpHRrYXJo\" class=\"ipyplot-content-div-ZmBbGKRVkcBs8r4hmraygy\">\n",
       "            <h4 style=\"font-size: 12px; word-wrap: break-word;\">\n",
       "                     Photo title: brown and black dogs running on snow   <br/>\n",
       "                     Distance: 0.73539\n",
       "            \n",
       "                     </h4>\n",
       "            <h4 style=\"font-size: 9px; padding-left: 10px; padding-right: 10px; width: 95%; word-wrap: break-word; white-space: normal;\">../data/raw/images/FAcSe7SjDUU.jpg</h4><img src=\"../data/raw/images/FAcSe7SjDUU.jpg\"/>\n",
       "            <a href=\"#!\">\n",
       "                <span class=\"ipyplot-img-close\"/>\n",
       "            </a>\n",
       "            <a href=\"#ipyplot-content-div-ZmBbGKRVkcBs8r4hmraygy-X4LCbG2mxoxuQtpHRrYXJo\">\n",
       "                <span class=\"ipyplot-img-expand\"/>\n",
       "            </a>\n",
       "        </div>\n",
       "    </div>\n",
       "    \n",
       "    <div class=\"ipyplot-placeholder-div-ZmBbGKRVkcBs8r4hmraygy\">\n",
       "        <div id=\"ipyplot-content-div-ZmBbGKRVkcBs8r4hmraygy-8bTHGURZxJ8WnbjqQvugsc\" class=\"ipyplot-content-div-ZmBbGKRVkcBs8r4hmraygy\">\n",
       "            <h4 style=\"font-size: 12px; word-wrap: break-word;\">\n",
       "                     Photo title: white and black dog on snow   <br/>\n",
       "                     Distance: 0.68213\n",
       "            \n",
       "                     </h4>\n",
       "            <h4 style=\"font-size: 9px; padding-left: 10px; padding-right: 10px; width: 95%; word-wrap: break-word; white-space: normal;\">../data/raw/images/jzCmSH6en5c.jpg</h4><img src=\"../data/raw/images/jzCmSH6en5c.jpg\"/>\n",
       "            <a href=\"#!\">\n",
       "                <span class=\"ipyplot-img-close\"/>\n",
       "            </a>\n",
       "            <a href=\"#ipyplot-content-div-ZmBbGKRVkcBs8r4hmraygy-8bTHGURZxJ8WnbjqQvugsc\">\n",
       "                <span class=\"ipyplot-img-expand\"/>\n",
       "            </a>\n",
       "        </div>\n",
       "    </div>\n",
       "    \n",
       "    <div class=\"ipyplot-placeholder-div-ZmBbGKRVkcBs8r4hmraygy\">\n",
       "        <div id=\"ipyplot-content-div-ZmBbGKRVkcBs8r4hmraygy-gc72YS83WXewnRzmmmGrM5\" class=\"ipyplot-content-div-ZmBbGKRVkcBs8r4hmraygy\">\n",
       "            <h4 style=\"font-size: 12px; word-wrap: break-word;\">\n",
       "                     Photo title: Dog in snow   <br/>\n",
       "                     Distance: 0.66102\n",
       "            \n",
       "                     </h4>\n",
       "            <h4 style=\"font-size: 9px; padding-left: 10px; padding-right: 10px; width: 95%; word-wrap: break-word; white-space: normal;\">../data/raw/images/jtnH16_HfPE.jpg</h4><img src=\"../data/raw/images/jtnH16_HfPE.jpg\"/>\n",
       "            <a href=\"#!\">\n",
       "                <span class=\"ipyplot-img-close\"/>\n",
       "            </a>\n",
       "            <a href=\"#ipyplot-content-div-ZmBbGKRVkcBs8r4hmraygy-gc72YS83WXewnRzmmmGrM5\">\n",
       "                <span class=\"ipyplot-img-expand\"/>\n",
       "            </a>\n",
       "        </div>\n",
       "    </div>\n",
       "    \n",
       "    <div class=\"ipyplot-placeholder-div-ZmBbGKRVkcBs8r4hmraygy\">\n",
       "        <div id=\"ipyplot-content-div-ZmBbGKRVkcBs8r4hmraygy-2wUUT8yeyDqe6b85mdodqm\" class=\"ipyplot-content-div-ZmBbGKRVkcBs8r4hmraygy\">\n",
       "            <h4 style=\"font-size: 12px; word-wrap: break-word;\">\n",
       "                     Photo title: tan dog playing on snow   <br/>\n",
       "                     Distance: 0.64610\n",
       "            \n",
       "                     </h4>\n",
       "            <h4 style=\"font-size: 9px; padding-left: 10px; padding-right: 10px; width: 95%; word-wrap: break-word; white-space: normal;\">../data/raw/images/AVUX8QXnj4Y.jpg</h4><img src=\"../data/raw/images/AVUX8QXnj4Y.jpg\"/>\n",
       "            <a href=\"#!\">\n",
       "                <span class=\"ipyplot-img-close\"/>\n",
       "            </a>\n",
       "            <a href=\"#ipyplot-content-div-ZmBbGKRVkcBs8r4hmraygy-2wUUT8yeyDqe6b85mdodqm\">\n",
       "                <span class=\"ipyplot-img-expand\"/>\n",
       "            </a>\n",
       "        </div>\n",
       "    </div>\n",
       "    \n",
       "    <div class=\"ipyplot-placeholder-div-ZmBbGKRVkcBs8r4hmraygy\">\n",
       "        <div id=\"ipyplot-content-div-ZmBbGKRVkcBs8r4hmraygy-9ZFf4397JbcBtHxLwhVR7X\" class=\"ipyplot-content-div-ZmBbGKRVkcBs8r4hmraygy\">\n",
       "            <h4 style=\"font-size: 12px; word-wrap: break-word;\">\n",
       "                     Photo title: white and black dog on snow field   <br/>\n",
       "                     Distance: 0.63227\n",
       "            \n",
       "                     </h4>\n",
       "            <h4 style=\"font-size: 9px; padding-left: 10px; padding-right: 10px; width: 95%; word-wrap: break-word; white-space: normal;\">../data/raw/images/QtxgNsmJQSs.jpg</h4><img src=\"../data/raw/images/QtxgNsmJQSs.jpg\"/>\n",
       "            <a href=\"#!\">\n",
       "                <span class=\"ipyplot-img-close\"/>\n",
       "            </a>\n",
       "            <a href=\"#ipyplot-content-div-ZmBbGKRVkcBs8r4hmraygy-9ZFf4397JbcBtHxLwhVR7X\">\n",
       "                <span class=\"ipyplot-img-expand\"/>\n",
       "            </a>\n",
       "        </div>\n",
       "    </div>\n",
       "    </div>\n",
       "        </textarea>\n",
       "    </div>\n",
       "    "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "\n",
       "        <style>\n",
       "        #ipyplot-imgs-container-div-ZmBbGKRVkcBs8r4hmraygy {\n",
       "            width: 100%;\n",
       "            height: 100%;\n",
       "            margin: 0%;\n",
       "            overflow: auto;\n",
       "            position: relative;\n",
       "            overflow-y: scroll;\n",
       "        }\n",
       "\n",
       "        div.ipyplot-placeholder-div-ZmBbGKRVkcBs8r4hmraygy {\n",
       "            width: 200px;\n",
       "            display: inline-block;\n",
       "            margin: 3px;\n",
       "            position: relative;\n",
       "        }\n",
       "\n",
       "        div.ipyplot-content-div-ZmBbGKRVkcBs8r4hmraygy {\n",
       "            width: 200px;\n",
       "            background: white;\n",
       "            display: inline-block;\n",
       "            vertical-align: top;\n",
       "            text-align: center;\n",
       "            position: relative;\n",
       "            border: 2px solid #ddd;\n",
       "            top: 0;\n",
       "            left: 0;\n",
       "        }\n",
       "\n",
       "        div.ipyplot-content-div-ZmBbGKRVkcBs8r4hmraygy span.ipyplot-img-close {\n",
       "            display: none;\n",
       "        }\n",
       "\n",
       "        div.ipyplot-content-div-ZmBbGKRVkcBs8r4hmraygy span {\n",
       "            width: 100%;\n",
       "            height: 100%;\n",
       "            position: absolute;\n",
       "            top: 0;\n",
       "            left: 0;\n",
       "        }\n",
       "\n",
       "        div.ipyplot-content-div-ZmBbGKRVkcBs8r4hmraygy img {\n",
       "            width: 200px;\n",
       "        }\n",
       "\n",
       "        div.ipyplot-content-div-ZmBbGKRVkcBs8r4hmraygy span.ipyplot-img-close:hover {\n",
       "            cursor: zoom-out;\n",
       "        }\n",
       "        div.ipyplot-content-div-ZmBbGKRVkcBs8r4hmraygy span.ipyplot-img-expand:hover {\n",
       "            cursor: zoom-in;\n",
       "        }\n",
       "\n",
       "        div[id^=ipyplot-content-div-ZmBbGKRVkcBs8r4hmraygy]:target {\n",
       "            transform: scale(2.5);\n",
       "            transform-origin: left top;\n",
       "            z-index: 5000;\n",
       "            top: 0;\n",
       "            left: 0;\n",
       "            position: absolute;\n",
       "        }\n",
       "\n",
       "        div[id^=ipyplot-content-div-ZmBbGKRVkcBs8r4hmraygy]:target span.ipyplot-img-close {\n",
       "            display: block;\n",
       "        }\n",
       "\n",
       "        div[id^=ipyplot-content-div-ZmBbGKRVkcBs8r4hmraygy]:target span.ipyplot-img-expand {\n",
       "            display: none;\n",
       "        }\n",
       "        </style>\n",
       "    <div id=\"ipyplot-imgs-container-div-ZmBbGKRVkcBs8r4hmraygy\">\n",
       "    <div class=\"ipyplot-placeholder-div-ZmBbGKRVkcBs8r4hmraygy\">\n",
       "        <div id=\"ipyplot-content-div-ZmBbGKRVkcBs8r4hmraygy-X4LCbG2mxoxuQtpHRrYXJo\" class=\"ipyplot-content-div-ZmBbGKRVkcBs8r4hmraygy\">\n",
       "            <h4 style=\"font-size: 12px; word-wrap: break-word;\">\n",
       "                     Photo title: brown and black dogs running on snow   <br/>\n",
       "                     Distance: 0.73539\n",
       "            \n",
       "                     </h4>\n",
       "            <h4 style=\"font-size: 9px; padding-left: 10px; padding-right: 10px; width: 95%; word-wrap: break-word; white-space: normal;\">../data/raw/images/FAcSe7SjDUU.jpg</h4><img src=\"../data/raw/images/FAcSe7SjDUU.jpg\"/>\n",
       "            <a href=\"#!\">\n",
       "                <span class=\"ipyplot-img-close\"/>\n",
       "            </a>\n",
       "            <a href=\"#ipyplot-content-div-ZmBbGKRVkcBs8r4hmraygy-X4LCbG2mxoxuQtpHRrYXJo\">\n",
       "                <span class=\"ipyplot-img-expand\"/>\n",
       "            </a>\n",
       "        </div>\n",
       "    </div>\n",
       "    \n",
       "    <div class=\"ipyplot-placeholder-div-ZmBbGKRVkcBs8r4hmraygy\">\n",
       "        <div id=\"ipyplot-content-div-ZmBbGKRVkcBs8r4hmraygy-8bTHGURZxJ8WnbjqQvugsc\" class=\"ipyplot-content-div-ZmBbGKRVkcBs8r4hmraygy\">\n",
       "            <h4 style=\"font-size: 12px; word-wrap: break-word;\">\n",
       "                     Photo title: white and black dog on snow   <br/>\n",
       "                     Distance: 0.68213\n",
       "            \n",
       "                     </h4>\n",
       "            <h4 style=\"font-size: 9px; padding-left: 10px; padding-right: 10px; width: 95%; word-wrap: break-word; white-space: normal;\">../data/raw/images/jzCmSH6en5c.jpg</h4><img src=\"../data/raw/images/jzCmSH6en5c.jpg\"/>\n",
       "            <a href=\"#!\">\n",
       "                <span class=\"ipyplot-img-close\"/>\n",
       "            </a>\n",
       "            <a href=\"#ipyplot-content-div-ZmBbGKRVkcBs8r4hmraygy-8bTHGURZxJ8WnbjqQvugsc\">\n",
       "                <span class=\"ipyplot-img-expand\"/>\n",
       "            </a>\n",
       "        </div>\n",
       "    </div>\n",
       "    \n",
       "    <div class=\"ipyplot-placeholder-div-ZmBbGKRVkcBs8r4hmraygy\">\n",
       "        <div id=\"ipyplot-content-div-ZmBbGKRVkcBs8r4hmraygy-gc72YS83WXewnRzmmmGrM5\" class=\"ipyplot-content-div-ZmBbGKRVkcBs8r4hmraygy\">\n",
       "            <h4 style=\"font-size: 12px; word-wrap: break-word;\">\n",
       "                     Photo title: Dog in snow   <br/>\n",
       "                     Distance: 0.66102\n",
       "            \n",
       "                     </h4>\n",
       "            <h4 style=\"font-size: 9px; padding-left: 10px; padding-right: 10px; width: 95%; word-wrap: break-word; white-space: normal;\">../data/raw/images/jtnH16_HfPE.jpg</h4><img src=\"../data/raw/images/jtnH16_HfPE.jpg\"/>\n",
       "            <a href=\"#!\">\n",
       "                <span class=\"ipyplot-img-close\"/>\n",
       "            </a>\n",
       "            <a href=\"#ipyplot-content-div-ZmBbGKRVkcBs8r4hmraygy-gc72YS83WXewnRzmmmGrM5\">\n",
       "                <span class=\"ipyplot-img-expand\"/>\n",
       "            </a>\n",
       "        </div>\n",
       "    </div>\n",
       "    \n",
       "    <div class=\"ipyplot-placeholder-div-ZmBbGKRVkcBs8r4hmraygy\">\n",
       "        <div id=\"ipyplot-content-div-ZmBbGKRVkcBs8r4hmraygy-2wUUT8yeyDqe6b85mdodqm\" class=\"ipyplot-content-div-ZmBbGKRVkcBs8r4hmraygy\">\n",
       "            <h4 style=\"font-size: 12px; word-wrap: break-word;\">\n",
       "                     Photo title: tan dog playing on snow   <br/>\n",
       "                     Distance: 0.64610\n",
       "            \n",
       "                     </h4>\n",
       "            <h4 style=\"font-size: 9px; padding-left: 10px; padding-right: 10px; width: 95%; word-wrap: break-word; white-space: normal;\">../data/raw/images/AVUX8QXnj4Y.jpg</h4><img src=\"../data/raw/images/AVUX8QXnj4Y.jpg\"/>\n",
       "            <a href=\"#!\">\n",
       "                <span class=\"ipyplot-img-close\"/>\n",
       "            </a>\n",
       "            <a href=\"#ipyplot-content-div-ZmBbGKRVkcBs8r4hmraygy-2wUUT8yeyDqe6b85mdodqm\">\n",
       "                <span class=\"ipyplot-img-expand\"/>\n",
       "            </a>\n",
       "        </div>\n",
       "    </div>\n",
       "    \n",
       "    <div class=\"ipyplot-placeholder-div-ZmBbGKRVkcBs8r4hmraygy\">\n",
       "        <div id=\"ipyplot-content-div-ZmBbGKRVkcBs8r4hmraygy-9ZFf4397JbcBtHxLwhVR7X\" class=\"ipyplot-content-div-ZmBbGKRVkcBs8r4hmraygy\">\n",
       "            <h4 style=\"font-size: 12px; word-wrap: break-word;\">\n",
       "                     Photo title: white and black dog on snow field   <br/>\n",
       "                     Distance: 0.63227\n",
       "            \n",
       "                     </h4>\n",
       "            <h4 style=\"font-size: 9px; padding-left: 10px; padding-right: 10px; width: 95%; word-wrap: break-word; white-space: normal;\">../data/raw/images/QtxgNsmJQSs.jpg</h4><img src=\"../data/raw/images/QtxgNsmJQSs.jpg\"/>\n",
       "            <a href=\"#!\">\n",
       "                <span class=\"ipyplot-img-close\"/>\n",
       "            </a>\n",
       "            <a href=\"#ipyplot-content-div-ZmBbGKRVkcBs8r4hmraygy-9ZFf4397JbcBtHxLwhVR7X\">\n",
       "                <span class=\"ipyplot-img-expand\"/>\n",
       "            </a>\n",
       "        </div>\n",
       "    </div>\n",
       "    </div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "find_results( \"Two dogs playing in the snow\", explain=False, k =5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "50fd5b84-1f6c-4ea6-935b-54d65fe48750",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "47f542ed-a26f-4dee-b998-ea6ad8d6aac9",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "id": "e4135300-f814-4995-b22c-bc83138bfac0",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-25T23:13:44.404152Z",
     "iopub.status.busy": "2023-04-25T23:13:44.403339Z",
     "iopub.status.idle": "2023-04-25T23:13:44.505260Z",
     "shell.execute_reply": "2023-04-25T23:13:44.503908Z",
     "shell.execute_reply.started": "2023-04-25T23:13:44.404101Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<h4>Query: boy and girl on a beach </h4>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['boy', 'and', 'girl', 'on', 'a', 'beach']\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <style>\n",
       "        #ipyplot-html-viewer-toggle-3Vt32rdLvGWLf7VMkr83g9 {\n",
       "            position: absolute;\n",
       "            top: -9999px;\n",
       "            left: -9999px;\n",
       "            visibility: hidden;\n",
       "        }\n",
       "\n",
       "        #ipyplot-html-viewer-label-3Vt32rdLvGWLf7VMkr83g9 { \n",
       "            position: relative;\n",
       "            display: inline-block;\n",
       "            cursor: pointer;\n",
       "            color: blue;\n",
       "            text-decoration: underline;\n",
       "        }\n",
       "\n",
       "        #ipyplot-html-viewer-textarea-3Vt32rdLvGWLf7VMkr83g9 {\n",
       "            background: lightgrey;\n",
       "            width: 100%;\n",
       "            height: 0px;\n",
       "            display: none;\n",
       "        }\n",
       "\n",
       "        #ipyplot-html-viewer-toggle-3Vt32rdLvGWLf7VMkr83g9:checked ~ #ipyplot-html-viewer-textarea-3Vt32rdLvGWLf7VMkr83g9 {\n",
       "            height: 200px;\n",
       "            display: block;\n",
       "        }\n",
       "\n",
       "        #ipyplot-html-viewer-toggle-3Vt32rdLvGWLf7VMkr83g9:checked + #ipyplot-html-viewer-label-3Vt32rdLvGWLf7VMkr83g9:after {\n",
       "            content: \"hide html\";\n",
       "            position: absolute;\n",
       "            top: 0;\n",
       "            left: 0;\n",
       "            right: 0;\n",
       "            bottom: 0;\n",
       "            background: white;\n",
       "            cursor: pointer;\n",
       "            color: blue;\n",
       "            text-decoration: underline;\n",
       "        }\n",
       "    </style>\n",
       "    <div>\n",
       "        <input type=\"checkbox\" id=\"ipyplot-html-viewer-toggle-3Vt32rdLvGWLf7VMkr83g9\">\n",
       "        <label id=\"ipyplot-html-viewer-label-3Vt32rdLvGWLf7VMkr83g9\" for=\"ipyplot-html-viewer-toggle-3Vt32rdLvGWLf7VMkr83g9\">show html</label>\n",
       "        <textarea id=\"ipyplot-html-viewer-textarea-3Vt32rdLvGWLf7VMkr83g9\" readonly>\n",
       "            \n",
       "        <style>\n",
       "        #ipyplot-imgs-container-div-TUUfNPDgetVWFFQKUVu5ii {\n",
       "            width: 100%;\n",
       "            height: 100%;\n",
       "            margin: 0%;\n",
       "            overflow: auto;\n",
       "            position: relative;\n",
       "            overflow-y: scroll;\n",
       "        }\n",
       "\n",
       "        div.ipyplot-placeholder-div-TUUfNPDgetVWFFQKUVu5ii {\n",
       "            width: 200px;\n",
       "            display: inline-block;\n",
       "            margin: 3px;\n",
       "            position: relative;\n",
       "        }\n",
       "\n",
       "        div.ipyplot-content-div-TUUfNPDgetVWFFQKUVu5ii {\n",
       "            width: 200px;\n",
       "            background: white;\n",
       "            display: inline-block;\n",
       "            vertical-align: top;\n",
       "            text-align: center;\n",
       "            position: relative;\n",
       "            border: 2px solid #ddd;\n",
       "            top: 0;\n",
       "            left: 0;\n",
       "        }\n",
       "\n",
       "        div.ipyplot-content-div-TUUfNPDgetVWFFQKUVu5ii span.ipyplot-img-close {\n",
       "            display: none;\n",
       "        }\n",
       "\n",
       "        div.ipyplot-content-div-TUUfNPDgetVWFFQKUVu5ii span {\n",
       "            width: 100%;\n",
       "            height: 100%;\n",
       "            position: absolute;\n",
       "            top: 0;\n",
       "            left: 0;\n",
       "        }\n",
       "\n",
       "        div.ipyplot-content-div-TUUfNPDgetVWFFQKUVu5ii img {\n",
       "            width: 200px;\n",
       "        }\n",
       "\n",
       "        div.ipyplot-content-div-TUUfNPDgetVWFFQKUVu5ii span.ipyplot-img-close:hover {\n",
       "            cursor: zoom-out;\n",
       "        }\n",
       "        div.ipyplot-content-div-TUUfNPDgetVWFFQKUVu5ii span.ipyplot-img-expand:hover {\n",
       "            cursor: zoom-in;\n",
       "        }\n",
       "\n",
       "        div[id^=ipyplot-content-div-TUUfNPDgetVWFFQKUVu5ii]:target {\n",
       "            transform: scale(2.5);\n",
       "            transform-origin: left top;\n",
       "            z-index: 5000;\n",
       "            top: 0;\n",
       "            left: 0;\n",
       "            position: absolute;\n",
       "        }\n",
       "\n",
       "        div[id^=ipyplot-content-div-TUUfNPDgetVWFFQKUVu5ii]:target span.ipyplot-img-close {\n",
       "            display: block;\n",
       "        }\n",
       "\n",
       "        div[id^=ipyplot-content-div-TUUfNPDgetVWFFQKUVu5ii]:target span.ipyplot-img-expand {\n",
       "            display: none;\n",
       "        }\n",
       "        </style>\n",
       "    <div id=\"ipyplot-imgs-container-div-TUUfNPDgetVWFFQKUVu5ii\">\n",
       "    <div class=\"ipyplot-placeholder-div-TUUfNPDgetVWFFQKUVu5ii\">\n",
       "        <div id=\"ipyplot-content-div-TUUfNPDgetVWFFQKUVu5ii-BkxC4R94krywbQkLPzivJD\" class=\"ipyplot-content-div-TUUfNPDgetVWFFQKUVu5ii\">\n",
       "            <h4 style=\"font-size: 12px; word-wrap: break-word;\">\n",
       "                     Photo title: children enjoying the beach   <br/>\n",
       "                     Distance: 0.70779\n",
       "            \n",
       "                     </h4>\n",
       "            <h4 style=\"font-size: 9px; padding-left: 10px; padding-right: 10px; width: 95%; word-wrap: break-word; white-space: normal;\">../data/raw/images/pV87YnElHow.jpg</h4><img src=\"../data/raw/images/pV87YnElHow.jpg\"/>\n",
       "            <a href=\"#!\">\n",
       "                <span class=\"ipyplot-img-close\"/>\n",
       "            </a>\n",
       "            <a href=\"#ipyplot-content-div-TUUfNPDgetVWFFQKUVu5ii-BkxC4R94krywbQkLPzivJD\">\n",
       "                <span class=\"ipyplot-img-expand\"/>\n",
       "            </a>\n",
       "        </div>\n",
       "    </div>\n",
       "    \n",
       "    <div class=\"ipyplot-placeholder-div-TUUfNPDgetVWFFQKUVu5ii\">\n",
       "        <div id=\"ipyplot-content-div-TUUfNPDgetVWFFQKUVu5ii-ntFSZGEVMcDkBade93xh8Z\" class=\"ipyplot-content-div-TUUfNPDgetVWFFQKUVu5ii\">\n",
       "            <h4 style=\"font-size: 12px; word-wrap: break-word;\">\n",
       "                     Photo title: Couple on the beach   <br/>\n",
       "                     Distance: 0.69817\n",
       "            \n",
       "                     </h4>\n",
       "            <h4 style=\"font-size: 9px; padding-left: 10px; padding-right: 10px; width: 95%; word-wrap: break-word; white-space: normal;\">../data/raw/images/rl7mUDEUmVE.jpg</h4><img src=\"../data/raw/images/rl7mUDEUmVE.jpg\"/>\n",
       "            <a href=\"#!\">\n",
       "                <span class=\"ipyplot-img-close\"/>\n",
       "            </a>\n",
       "            <a href=\"#ipyplot-content-div-TUUfNPDgetVWFFQKUVu5ii-ntFSZGEVMcDkBade93xh8Z\">\n",
       "                <span class=\"ipyplot-img-expand\"/>\n",
       "            </a>\n",
       "        </div>\n",
       "    </div>\n",
       "    \n",
       "    <div class=\"ipyplot-placeholder-div-TUUfNPDgetVWFFQKUVu5ii\">\n",
       "        <div id=\"ipyplot-content-div-TUUfNPDgetVWFFQKUVu5ii-agsE9Fm3FGe6JAVpyi9vih\" class=\"ipyplot-content-div-TUUfNPDgetVWFFQKUVu5ii\">\n",
       "            <h4 style=\"font-size: 12px; word-wrap: break-word;\">\n",
       "                     Photo title: people at beach   <br/>\n",
       "                     Distance: 0.68724\n",
       "            \n",
       "                     </h4>\n",
       "            <h4 style=\"font-size: 9px; padding-left: 10px; padding-right: 10px; width: 95%; word-wrap: break-word; white-space: normal;\">../data/raw/images/DtHchyQtyZ8.jpg</h4><img src=\"../data/raw/images/DtHchyQtyZ8.jpg\"/>\n",
       "            <a href=\"#!\">\n",
       "                <span class=\"ipyplot-img-close\"/>\n",
       "            </a>\n",
       "            <a href=\"#ipyplot-content-div-TUUfNPDgetVWFFQKUVu5ii-agsE9Fm3FGe6JAVpyi9vih\">\n",
       "                <span class=\"ipyplot-img-expand\"/>\n",
       "            </a>\n",
       "        </div>\n",
       "    </div>\n",
       "    \n",
       "    <div class=\"ipyplot-placeholder-div-TUUfNPDgetVWFFQKUVu5ii\">\n",
       "        <div id=\"ipyplot-content-div-TUUfNPDgetVWFFQKUVu5ii-9EfBjfBHr6atBMEUPefqpq\" class=\"ipyplot-content-div-TUUfNPDgetVWFFQKUVu5ii\">\n",
       "            <h4 style=\"font-size: 12px; word-wrap: break-word;\">\n",
       "                     Photo title: two people on beach during daytime   <br/>\n",
       "                     Distance: 0.67832\n",
       "            \n",
       "                     </h4>\n",
       "            <h4 style=\"font-size: 9px; padding-left: 10px; padding-right: 10px; width: 95%; word-wrap: break-word; white-space: normal;\">../data/raw/images/ejhtQmbGZr8.jpg</h4><img src=\"../data/raw/images/ejhtQmbGZr8.jpg\"/>\n",
       "            <a href=\"#!\">\n",
       "                <span class=\"ipyplot-img-close\"/>\n",
       "            </a>\n",
       "            <a href=\"#ipyplot-content-div-TUUfNPDgetVWFFQKUVu5ii-9EfBjfBHr6atBMEUPefqpq\">\n",
       "                <span class=\"ipyplot-img-expand\"/>\n",
       "            </a>\n",
       "        </div>\n",
       "    </div>\n",
       "    \n",
       "    <div class=\"ipyplot-placeholder-div-TUUfNPDgetVWFFQKUVu5ii\">\n",
       "        <div id=\"ipyplot-content-div-TUUfNPDgetVWFFQKUVu5ii-93Ni8wjStXevkP2CYt4b7p\" class=\"ipyplot-content-div-TUUfNPDgetVWFFQKUVu5ii\">\n",
       "            <h4 style=\"font-size: 12px; word-wrap: break-word;\">\n",
       "                     Photo title: A black and white beach   <br/>\n",
       "                     Distance: 0.59912\n",
       "            \n",
       "                     </h4>\n",
       "            <h4 style=\"font-size: 9px; padding-left: 10px; padding-right: 10px; width: 95%; word-wrap: break-word; white-space: normal;\">../data/raw/images/wra9MptZ_ag.jpg</h4><img src=\"../data/raw/images/wra9MptZ_ag.jpg\"/>\n",
       "            <a href=\"#!\">\n",
       "                <span class=\"ipyplot-img-close\"/>\n",
       "            </a>\n",
       "            <a href=\"#ipyplot-content-div-TUUfNPDgetVWFFQKUVu5ii-93Ni8wjStXevkP2CYt4b7p\">\n",
       "                <span class=\"ipyplot-img-expand\"/>\n",
       "            </a>\n",
       "        </div>\n",
       "    </div>\n",
       "    </div>\n",
       "        </textarea>\n",
       "    </div>\n",
       "    "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "\n",
       "        <style>\n",
       "        #ipyplot-imgs-container-div-TUUfNPDgetVWFFQKUVu5ii {\n",
       "            width: 100%;\n",
       "            height: 100%;\n",
       "            margin: 0%;\n",
       "            overflow: auto;\n",
       "            position: relative;\n",
       "            overflow-y: scroll;\n",
       "        }\n",
       "\n",
       "        div.ipyplot-placeholder-div-TUUfNPDgetVWFFQKUVu5ii {\n",
       "            width: 200px;\n",
       "            display: inline-block;\n",
       "            margin: 3px;\n",
       "            position: relative;\n",
       "        }\n",
       "\n",
       "        div.ipyplot-content-div-TUUfNPDgetVWFFQKUVu5ii {\n",
       "            width: 200px;\n",
       "            background: white;\n",
       "            display: inline-block;\n",
       "            vertical-align: top;\n",
       "            text-align: center;\n",
       "            position: relative;\n",
       "            border: 2px solid #ddd;\n",
       "            top: 0;\n",
       "            left: 0;\n",
       "        }\n",
       "\n",
       "        div.ipyplot-content-div-TUUfNPDgetVWFFQKUVu5ii span.ipyplot-img-close {\n",
       "            display: none;\n",
       "        }\n",
       "\n",
       "        div.ipyplot-content-div-TUUfNPDgetVWFFQKUVu5ii span {\n",
       "            width: 100%;\n",
       "            height: 100%;\n",
       "            position: absolute;\n",
       "            top: 0;\n",
       "            left: 0;\n",
       "        }\n",
       "\n",
       "        div.ipyplot-content-div-TUUfNPDgetVWFFQKUVu5ii img {\n",
       "            width: 200px;\n",
       "        }\n",
       "\n",
       "        div.ipyplot-content-div-TUUfNPDgetVWFFQKUVu5ii span.ipyplot-img-close:hover {\n",
       "            cursor: zoom-out;\n",
       "        }\n",
       "        div.ipyplot-content-div-TUUfNPDgetVWFFQKUVu5ii span.ipyplot-img-expand:hover {\n",
       "            cursor: zoom-in;\n",
       "        }\n",
       "\n",
       "        div[id^=ipyplot-content-div-TUUfNPDgetVWFFQKUVu5ii]:target {\n",
       "            transform: scale(2.5);\n",
       "            transform-origin: left top;\n",
       "            z-index: 5000;\n",
       "            top: 0;\n",
       "            left: 0;\n",
       "            position: absolute;\n",
       "        }\n",
       "\n",
       "        div[id^=ipyplot-content-div-TUUfNPDgetVWFFQKUVu5ii]:target span.ipyplot-img-close {\n",
       "            display: block;\n",
       "        }\n",
       "\n",
       "        div[id^=ipyplot-content-div-TUUfNPDgetVWFFQKUVu5ii]:target span.ipyplot-img-expand {\n",
       "            display: none;\n",
       "        }\n",
       "        </style>\n",
       "    <div id=\"ipyplot-imgs-container-div-TUUfNPDgetVWFFQKUVu5ii\">\n",
       "    <div class=\"ipyplot-placeholder-div-TUUfNPDgetVWFFQKUVu5ii\">\n",
       "        <div id=\"ipyplot-content-div-TUUfNPDgetVWFFQKUVu5ii-BkxC4R94krywbQkLPzivJD\" class=\"ipyplot-content-div-TUUfNPDgetVWFFQKUVu5ii\">\n",
       "            <h4 style=\"font-size: 12px; word-wrap: break-word;\">\n",
       "                     Photo title: children enjoying the beach   <br/>\n",
       "                     Distance: 0.70779\n",
       "            \n",
       "                     </h4>\n",
       "            <h4 style=\"font-size: 9px; padding-left: 10px; padding-right: 10px; width: 95%; word-wrap: break-word; white-space: normal;\">../data/raw/images/pV87YnElHow.jpg</h4><img src=\"../data/raw/images/pV87YnElHow.jpg\"/>\n",
       "            <a href=\"#!\">\n",
       "                <span class=\"ipyplot-img-close\"/>\n",
       "            </a>\n",
       "            <a href=\"#ipyplot-content-div-TUUfNPDgetVWFFQKUVu5ii-BkxC4R94krywbQkLPzivJD\">\n",
       "                <span class=\"ipyplot-img-expand\"/>\n",
       "            </a>\n",
       "        </div>\n",
       "    </div>\n",
       "    \n",
       "    <div class=\"ipyplot-placeholder-div-TUUfNPDgetVWFFQKUVu5ii\">\n",
       "        <div id=\"ipyplot-content-div-TUUfNPDgetVWFFQKUVu5ii-ntFSZGEVMcDkBade93xh8Z\" class=\"ipyplot-content-div-TUUfNPDgetVWFFQKUVu5ii\">\n",
       "            <h4 style=\"font-size: 12px; word-wrap: break-word;\">\n",
       "                     Photo title: Couple on the beach   <br/>\n",
       "                     Distance: 0.69817\n",
       "            \n",
       "                     </h4>\n",
       "            <h4 style=\"font-size: 9px; padding-left: 10px; padding-right: 10px; width: 95%; word-wrap: break-word; white-space: normal;\">../data/raw/images/rl7mUDEUmVE.jpg</h4><img src=\"../data/raw/images/rl7mUDEUmVE.jpg\"/>\n",
       "            <a href=\"#!\">\n",
       "                <span class=\"ipyplot-img-close\"/>\n",
       "            </a>\n",
       "            <a href=\"#ipyplot-content-div-TUUfNPDgetVWFFQKUVu5ii-ntFSZGEVMcDkBade93xh8Z\">\n",
       "                <span class=\"ipyplot-img-expand\"/>\n",
       "            </a>\n",
       "        </div>\n",
       "    </div>\n",
       "    \n",
       "    <div class=\"ipyplot-placeholder-div-TUUfNPDgetVWFFQKUVu5ii\">\n",
       "        <div id=\"ipyplot-content-div-TUUfNPDgetVWFFQKUVu5ii-agsE9Fm3FGe6JAVpyi9vih\" class=\"ipyplot-content-div-TUUfNPDgetVWFFQKUVu5ii\">\n",
       "            <h4 style=\"font-size: 12px; word-wrap: break-word;\">\n",
       "                     Photo title: people at beach   <br/>\n",
       "                     Distance: 0.68724\n",
       "            \n",
       "                     </h4>\n",
       "            <h4 style=\"font-size: 9px; padding-left: 10px; padding-right: 10px; width: 95%; word-wrap: break-word; white-space: normal;\">../data/raw/images/DtHchyQtyZ8.jpg</h4><img src=\"../data/raw/images/DtHchyQtyZ8.jpg\"/>\n",
       "            <a href=\"#!\">\n",
       "                <span class=\"ipyplot-img-close\"/>\n",
       "            </a>\n",
       "            <a href=\"#ipyplot-content-div-TUUfNPDgetVWFFQKUVu5ii-agsE9Fm3FGe6JAVpyi9vih\">\n",
       "                <span class=\"ipyplot-img-expand\"/>\n",
       "            </a>\n",
       "        </div>\n",
       "    </div>\n",
       "    \n",
       "    <div class=\"ipyplot-placeholder-div-TUUfNPDgetVWFFQKUVu5ii\">\n",
       "        <div id=\"ipyplot-content-div-TUUfNPDgetVWFFQKUVu5ii-9EfBjfBHr6atBMEUPefqpq\" class=\"ipyplot-content-div-TUUfNPDgetVWFFQKUVu5ii\">\n",
       "            <h4 style=\"font-size: 12px; word-wrap: break-word;\">\n",
       "                     Photo title: two people on beach during daytime   <br/>\n",
       "                     Distance: 0.67832\n",
       "            \n",
       "                     </h4>\n",
       "            <h4 style=\"font-size: 9px; padding-left: 10px; padding-right: 10px; width: 95%; word-wrap: break-word; white-space: normal;\">../data/raw/images/ejhtQmbGZr8.jpg</h4><img src=\"../data/raw/images/ejhtQmbGZr8.jpg\"/>\n",
       "            <a href=\"#!\">\n",
       "                <span class=\"ipyplot-img-close\"/>\n",
       "            </a>\n",
       "            <a href=\"#ipyplot-content-div-TUUfNPDgetVWFFQKUVu5ii-9EfBjfBHr6atBMEUPefqpq\">\n",
       "                <span class=\"ipyplot-img-expand\"/>\n",
       "            </a>\n",
       "        </div>\n",
       "    </div>\n",
       "    \n",
       "    <div class=\"ipyplot-placeholder-div-TUUfNPDgetVWFFQKUVu5ii\">\n",
       "        <div id=\"ipyplot-content-div-TUUfNPDgetVWFFQKUVu5ii-93Ni8wjStXevkP2CYt4b7p\" class=\"ipyplot-content-div-TUUfNPDgetVWFFQKUVu5ii\">\n",
       "            <h4 style=\"font-size: 12px; word-wrap: break-word;\">\n",
       "                     Photo title: A black and white beach   <br/>\n",
       "                     Distance: 0.59912\n",
       "            \n",
       "                     </h4>\n",
       "            <h4 style=\"font-size: 9px; padding-left: 10px; padding-right: 10px; width: 95%; word-wrap: break-word; white-space: normal;\">../data/raw/images/wra9MptZ_ag.jpg</h4><img src=\"../data/raw/images/wra9MptZ_ag.jpg\"/>\n",
       "            <a href=\"#!\">\n",
       "                <span class=\"ipyplot-img-close\"/>\n",
       "            </a>\n",
       "            <a href=\"#ipyplot-content-div-TUUfNPDgetVWFFQKUVu5ii-93Ni8wjStXevkP2CYt4b7p\">\n",
       "                <span class=\"ipyplot-img-expand\"/>\n",
       "            </a>\n",
       "        </div>\n",
       "    </div>\n",
       "    </div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "find_results( \"boy and girl on a beach\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7d22bd79-da3b-4442-abbf-4e469b6cfdcb",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "id": "96b8222c-a46f-4ea2-bf48-11fde5f781ae",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-25T23:13:55.576720Z",
     "iopub.status.busy": "2023-04-25T23:13:55.575889Z",
     "iopub.status.idle": "2023-04-25T23:13:55.659363Z",
     "shell.execute_reply": "2023-04-25T23:13:55.657899Z",
     "shell.execute_reply.started": "2023-04-25T23:13:55.576686Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<h4>Query: image of a man in a desert </h4>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['image', 'of', 'a', 'man', 'in', 'a', 'desert']\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <style>\n",
       "        #ipyplot-html-viewer-toggle-3HYaZrcpNLhmyKXx4pm8D5 {\n",
       "            position: absolute;\n",
       "            top: -9999px;\n",
       "            left: -9999px;\n",
       "            visibility: hidden;\n",
       "        }\n",
       "\n",
       "        #ipyplot-html-viewer-label-3HYaZrcpNLhmyKXx4pm8D5 { \n",
       "            position: relative;\n",
       "            display: inline-block;\n",
       "            cursor: pointer;\n",
       "            color: blue;\n",
       "            text-decoration: underline;\n",
       "        }\n",
       "\n",
       "        #ipyplot-html-viewer-textarea-3HYaZrcpNLhmyKXx4pm8D5 {\n",
       "            background: lightgrey;\n",
       "            width: 100%;\n",
       "            height: 0px;\n",
       "            display: none;\n",
       "        }\n",
       "\n",
       "        #ipyplot-html-viewer-toggle-3HYaZrcpNLhmyKXx4pm8D5:checked ~ #ipyplot-html-viewer-textarea-3HYaZrcpNLhmyKXx4pm8D5 {\n",
       "            height: 200px;\n",
       "            display: block;\n",
       "        }\n",
       "\n",
       "        #ipyplot-html-viewer-toggle-3HYaZrcpNLhmyKXx4pm8D5:checked + #ipyplot-html-viewer-label-3HYaZrcpNLhmyKXx4pm8D5:after {\n",
       "            content: \"hide html\";\n",
       "            position: absolute;\n",
       "            top: 0;\n",
       "            left: 0;\n",
       "            right: 0;\n",
       "            bottom: 0;\n",
       "            background: white;\n",
       "            cursor: pointer;\n",
       "            color: blue;\n",
       "            text-decoration: underline;\n",
       "        }\n",
       "    </style>\n",
       "    <div>\n",
       "        <input type=\"checkbox\" id=\"ipyplot-html-viewer-toggle-3HYaZrcpNLhmyKXx4pm8D5\">\n",
       "        <label id=\"ipyplot-html-viewer-label-3HYaZrcpNLhmyKXx4pm8D5\" for=\"ipyplot-html-viewer-toggle-3HYaZrcpNLhmyKXx4pm8D5\">show html</label>\n",
       "        <textarea id=\"ipyplot-html-viewer-textarea-3HYaZrcpNLhmyKXx4pm8D5\" readonly>\n",
       "            \n",
       "        <style>\n",
       "        #ipyplot-imgs-container-div-ZRkj89Wb9F2srHpgBgQKAH {\n",
       "            width: 100%;\n",
       "            height: 100%;\n",
       "            margin: 0%;\n",
       "            overflow: auto;\n",
       "            position: relative;\n",
       "            overflow-y: scroll;\n",
       "        }\n",
       "\n",
       "        div.ipyplot-placeholder-div-ZRkj89Wb9F2srHpgBgQKAH {\n",
       "            width: 200px;\n",
       "            display: inline-block;\n",
       "            margin: 3px;\n",
       "            position: relative;\n",
       "        }\n",
       "\n",
       "        div.ipyplot-content-div-ZRkj89Wb9F2srHpgBgQKAH {\n",
       "            width: 200px;\n",
       "            background: white;\n",
       "            display: inline-block;\n",
       "            vertical-align: top;\n",
       "            text-align: center;\n",
       "            position: relative;\n",
       "            border: 2px solid #ddd;\n",
       "            top: 0;\n",
       "            left: 0;\n",
       "        }\n",
       "\n",
       "        div.ipyplot-content-div-ZRkj89Wb9F2srHpgBgQKAH span.ipyplot-img-close {\n",
       "            display: none;\n",
       "        }\n",
       "\n",
       "        div.ipyplot-content-div-ZRkj89Wb9F2srHpgBgQKAH span {\n",
       "            width: 100%;\n",
       "            height: 100%;\n",
       "            position: absolute;\n",
       "            top: 0;\n",
       "            left: 0;\n",
       "        }\n",
       "\n",
       "        div.ipyplot-content-div-ZRkj89Wb9F2srHpgBgQKAH img {\n",
       "            width: 200px;\n",
       "        }\n",
       "\n",
       "        div.ipyplot-content-div-ZRkj89Wb9F2srHpgBgQKAH span.ipyplot-img-close:hover {\n",
       "            cursor: zoom-out;\n",
       "        }\n",
       "        div.ipyplot-content-div-ZRkj89Wb9F2srHpgBgQKAH span.ipyplot-img-expand:hover {\n",
       "            cursor: zoom-in;\n",
       "        }\n",
       "\n",
       "        div[id^=ipyplot-content-div-ZRkj89Wb9F2srHpgBgQKAH]:target {\n",
       "            transform: scale(2.5);\n",
       "            transform-origin: left top;\n",
       "            z-index: 5000;\n",
       "            top: 0;\n",
       "            left: 0;\n",
       "            position: absolute;\n",
       "        }\n",
       "\n",
       "        div[id^=ipyplot-content-div-ZRkj89Wb9F2srHpgBgQKAH]:target span.ipyplot-img-close {\n",
       "            display: block;\n",
       "        }\n",
       "\n",
       "        div[id^=ipyplot-content-div-ZRkj89Wb9F2srHpgBgQKAH]:target span.ipyplot-img-expand {\n",
       "            display: none;\n",
       "        }\n",
       "        </style>\n",
       "    <div id=\"ipyplot-imgs-container-div-ZRkj89Wb9F2srHpgBgQKAH\">\n",
       "    <div class=\"ipyplot-placeholder-div-ZRkj89Wb9F2srHpgBgQKAH\">\n",
       "        <div id=\"ipyplot-content-div-ZRkj89Wb9F2srHpgBgQKAH-5frsSk6CAJTQyMfN2ZxfUN\" class=\"ipyplot-content-div-ZRkj89Wb9F2srHpgBgQKAH\">\n",
       "            <h4 style=\"font-size: 12px; word-wrap: break-word;\">\n",
       "                     Photo title: person on desert   <br/>\n",
       "                     Distance: 0.78134\n",
       "            \n",
       "                     </h4>\n",
       "            <h4 style=\"font-size: 9px; padding-left: 10px; padding-right: 10px; width: 95%; word-wrap: break-word; white-space: normal;\">../data/raw/images/ovXu5yiEVWc.jpg</h4><img src=\"../data/raw/images/ovXu5yiEVWc.jpg\"/>\n",
       "            <a href=\"#!\">\n",
       "                <span class=\"ipyplot-img-close\"/>\n",
       "            </a>\n",
       "            <a href=\"#ipyplot-content-div-ZRkj89Wb9F2srHpgBgQKAH-5frsSk6CAJTQyMfN2ZxfUN\">\n",
       "                <span class=\"ipyplot-img-expand\"/>\n",
       "            </a>\n",
       "        </div>\n",
       "    </div>\n",
       "    \n",
       "    <div class=\"ipyplot-placeholder-div-ZRkj89Wb9F2srHpgBgQKAH\">\n",
       "        <div id=\"ipyplot-content-div-ZRkj89Wb9F2srHpgBgQKAH-E6mcB9XCMK8X2Lr9mzFWmx\" class=\"ipyplot-content-div-ZRkj89Wb9F2srHpgBgQKAH\">\n",
       "            <h4 style=\"font-size: 12px; word-wrap: break-word;\">\n",
       "                     Photo title: landscape photo of a desert   <br/>\n",
       "                     Distance: 0.74381\n",
       "            \n",
       "                     </h4>\n",
       "            <h4 style=\"font-size: 9px; padding-left: 10px; padding-right: 10px; width: 95%; word-wrap: break-word; white-space: normal;\">../data/raw/images/rQjbpvDtJoU.jpg</h4><img src=\"../data/raw/images/rQjbpvDtJoU.jpg\"/>\n",
       "            <a href=\"#!\">\n",
       "                <span class=\"ipyplot-img-close\"/>\n",
       "            </a>\n",
       "            <a href=\"#ipyplot-content-div-ZRkj89Wb9F2srHpgBgQKAH-E6mcB9XCMK8X2Lr9mzFWmx\">\n",
       "                <span class=\"ipyplot-img-expand\"/>\n",
       "            </a>\n",
       "        </div>\n",
       "    </div>\n",
       "    \n",
       "    <div class=\"ipyplot-placeholder-div-ZRkj89Wb9F2srHpgBgQKAH\">\n",
       "        <div id=\"ipyplot-content-div-ZRkj89Wb9F2srHpgBgQKAH-98s2DvPaPYLPGuRQmNH252\" class=\"ipyplot-content-div-ZRkj89Wb9F2srHpgBgQKAH\">\n",
       "            <h4 style=\"font-size: 12px; word-wrap: break-word;\">\n",
       "                     Photo title: man standing at desert   <br/>\n",
       "                     Distance: 0.73866\n",
       "            \n",
       "                     </h4>\n",
       "            <h4 style=\"font-size: 9px; padding-left: 10px; padding-right: 10px; width: 95%; word-wrap: break-word; white-space: normal;\">../data/raw/images/eUSfRwB_qY8.jpg</h4><img src=\"../data/raw/images/eUSfRwB_qY8.jpg\"/>\n",
       "            <a href=\"#!\">\n",
       "                <span class=\"ipyplot-img-close\"/>\n",
       "            </a>\n",
       "            <a href=\"#ipyplot-content-div-ZRkj89Wb9F2srHpgBgQKAH-98s2DvPaPYLPGuRQmNH252\">\n",
       "                <span class=\"ipyplot-img-expand\"/>\n",
       "            </a>\n",
       "        </div>\n",
       "    </div>\n",
       "    \n",
       "    <div class=\"ipyplot-placeholder-div-ZRkj89Wb9F2srHpgBgQKAH\">\n",
       "        <div id=\"ipyplot-content-div-ZRkj89Wb9F2srHpgBgQKAH-N52oVHd43bVaYD6rbWNPpi\" class=\"ipyplot-content-div-ZRkj89Wb9F2srHpgBgQKAH\">\n",
       "            <h4 style=\"font-size: 12px; word-wrap: break-word;\">\n",
       "                     Photo title: grayscale photo of a desert   <br/>\n",
       "                     Distance: 0.73768\n",
       "            \n",
       "                     </h4>\n",
       "            <h4 style=\"font-size: 9px; padding-left: 10px; padding-right: 10px; width: 95%; word-wrap: break-word; white-space: normal;\">../data/raw/images/h0fDDhf-MWw.jpg</h4><img src=\"../data/raw/images/h0fDDhf-MWw.jpg\"/>\n",
       "            <a href=\"#!\">\n",
       "                <span class=\"ipyplot-img-close\"/>\n",
       "            </a>\n",
       "            <a href=\"#ipyplot-content-div-ZRkj89Wb9F2srHpgBgQKAH-N52oVHd43bVaYD6rbWNPpi\">\n",
       "                <span class=\"ipyplot-img-expand\"/>\n",
       "            </a>\n",
       "        </div>\n",
       "    </div>\n",
       "    \n",
       "    <div class=\"ipyplot-placeholder-div-ZRkj89Wb9F2srHpgBgQKAH\">\n",
       "        <div id=\"ipyplot-content-div-ZRkj89Wb9F2srHpgBgQKAH-cPSWpHNp4WWaAmpMztH8Wr\" class=\"ipyplot-content-div-ZRkj89Wb9F2srHpgBgQKAH\">\n",
       "            <h4 style=\"font-size: 12px; word-wrap: break-word;\">\n",
       "                     Photo title: silhouette of person standing on desert   <br/>\n",
       "                     Distance: 0.72608\n",
       "            \n",
       "                     </h4>\n",
       "            <h4 style=\"font-size: 9px; padding-left: 10px; padding-right: 10px; width: 95%; word-wrap: break-word; white-space: normal;\">../data/raw/images/p8BM14LpLF0.jpg</h4><img src=\"../data/raw/images/p8BM14LpLF0.jpg\"/>\n",
       "            <a href=\"#!\">\n",
       "                <span class=\"ipyplot-img-close\"/>\n",
       "            </a>\n",
       "            <a href=\"#ipyplot-content-div-ZRkj89Wb9F2srHpgBgQKAH-cPSWpHNp4WWaAmpMztH8Wr\">\n",
       "                <span class=\"ipyplot-img-expand\"/>\n",
       "            </a>\n",
       "        </div>\n",
       "    </div>\n",
       "    </div>\n",
       "        </textarea>\n",
       "    </div>\n",
       "    "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "\n",
       "        <style>\n",
       "        #ipyplot-imgs-container-div-ZRkj89Wb9F2srHpgBgQKAH {\n",
       "            width: 100%;\n",
       "            height: 100%;\n",
       "            margin: 0%;\n",
       "            overflow: auto;\n",
       "            position: relative;\n",
       "            overflow-y: scroll;\n",
       "        }\n",
       "\n",
       "        div.ipyplot-placeholder-div-ZRkj89Wb9F2srHpgBgQKAH {\n",
       "            width: 200px;\n",
       "            display: inline-block;\n",
       "            margin: 3px;\n",
       "            position: relative;\n",
       "        }\n",
       "\n",
       "        div.ipyplot-content-div-ZRkj89Wb9F2srHpgBgQKAH {\n",
       "            width: 200px;\n",
       "            background: white;\n",
       "            display: inline-block;\n",
       "            vertical-align: top;\n",
       "            text-align: center;\n",
       "            position: relative;\n",
       "            border: 2px solid #ddd;\n",
       "            top: 0;\n",
       "            left: 0;\n",
       "        }\n",
       "\n",
       "        div.ipyplot-content-div-ZRkj89Wb9F2srHpgBgQKAH span.ipyplot-img-close {\n",
       "            display: none;\n",
       "        }\n",
       "\n",
       "        div.ipyplot-content-div-ZRkj89Wb9F2srHpgBgQKAH span {\n",
       "            width: 100%;\n",
       "            height: 100%;\n",
       "            position: absolute;\n",
       "            top: 0;\n",
       "            left: 0;\n",
       "        }\n",
       "\n",
       "        div.ipyplot-content-div-ZRkj89Wb9F2srHpgBgQKAH img {\n",
       "            width: 200px;\n",
       "        }\n",
       "\n",
       "        div.ipyplot-content-div-ZRkj89Wb9F2srHpgBgQKAH span.ipyplot-img-close:hover {\n",
       "            cursor: zoom-out;\n",
       "        }\n",
       "        div.ipyplot-content-div-ZRkj89Wb9F2srHpgBgQKAH span.ipyplot-img-expand:hover {\n",
       "            cursor: zoom-in;\n",
       "        }\n",
       "\n",
       "        div[id^=ipyplot-content-div-ZRkj89Wb9F2srHpgBgQKAH]:target {\n",
       "            transform: scale(2.5);\n",
       "            transform-origin: left top;\n",
       "            z-index: 5000;\n",
       "            top: 0;\n",
       "            left: 0;\n",
       "            position: absolute;\n",
       "        }\n",
       "\n",
       "        div[id^=ipyplot-content-div-ZRkj89Wb9F2srHpgBgQKAH]:target span.ipyplot-img-close {\n",
       "            display: block;\n",
       "        }\n",
       "\n",
       "        div[id^=ipyplot-content-div-ZRkj89Wb9F2srHpgBgQKAH]:target span.ipyplot-img-expand {\n",
       "            display: none;\n",
       "        }\n",
       "        </style>\n",
       "    <div id=\"ipyplot-imgs-container-div-ZRkj89Wb9F2srHpgBgQKAH\">\n",
       "    <div class=\"ipyplot-placeholder-div-ZRkj89Wb9F2srHpgBgQKAH\">\n",
       "        <div id=\"ipyplot-content-div-ZRkj89Wb9F2srHpgBgQKAH-5frsSk6CAJTQyMfN2ZxfUN\" class=\"ipyplot-content-div-ZRkj89Wb9F2srHpgBgQKAH\">\n",
       "            <h4 style=\"font-size: 12px; word-wrap: break-word;\">\n",
       "                     Photo title: person on desert   <br/>\n",
       "                     Distance: 0.78134\n",
       "            \n",
       "                     </h4>\n",
       "            <h4 style=\"font-size: 9px; padding-left: 10px; padding-right: 10px; width: 95%; word-wrap: break-word; white-space: normal;\">../data/raw/images/ovXu5yiEVWc.jpg</h4><img src=\"../data/raw/images/ovXu5yiEVWc.jpg\"/>\n",
       "            <a href=\"#!\">\n",
       "                <span class=\"ipyplot-img-close\"/>\n",
       "            </a>\n",
       "            <a href=\"#ipyplot-content-div-ZRkj89Wb9F2srHpgBgQKAH-5frsSk6CAJTQyMfN2ZxfUN\">\n",
       "                <span class=\"ipyplot-img-expand\"/>\n",
       "            </a>\n",
       "        </div>\n",
       "    </div>\n",
       "    \n",
       "    <div class=\"ipyplot-placeholder-div-ZRkj89Wb9F2srHpgBgQKAH\">\n",
       "        <div id=\"ipyplot-content-div-ZRkj89Wb9F2srHpgBgQKAH-E6mcB9XCMK8X2Lr9mzFWmx\" class=\"ipyplot-content-div-ZRkj89Wb9F2srHpgBgQKAH\">\n",
       "            <h4 style=\"font-size: 12px; word-wrap: break-word;\">\n",
       "                     Photo title: landscape photo of a desert   <br/>\n",
       "                     Distance: 0.74381\n",
       "            \n",
       "                     </h4>\n",
       "            <h4 style=\"font-size: 9px; padding-left: 10px; padding-right: 10px; width: 95%; word-wrap: break-word; white-space: normal;\">../data/raw/images/rQjbpvDtJoU.jpg</h4><img src=\"../data/raw/images/rQjbpvDtJoU.jpg\"/>\n",
       "            <a href=\"#!\">\n",
       "                <span class=\"ipyplot-img-close\"/>\n",
       "            </a>\n",
       "            <a href=\"#ipyplot-content-div-ZRkj89Wb9F2srHpgBgQKAH-E6mcB9XCMK8X2Lr9mzFWmx\">\n",
       "                <span class=\"ipyplot-img-expand\"/>\n",
       "            </a>\n",
       "        </div>\n",
       "    </div>\n",
       "    \n",
       "    <div class=\"ipyplot-placeholder-div-ZRkj89Wb9F2srHpgBgQKAH\">\n",
       "        <div id=\"ipyplot-content-div-ZRkj89Wb9F2srHpgBgQKAH-98s2DvPaPYLPGuRQmNH252\" class=\"ipyplot-content-div-ZRkj89Wb9F2srHpgBgQKAH\">\n",
       "            <h4 style=\"font-size: 12px; word-wrap: break-word;\">\n",
       "                     Photo title: man standing at desert   <br/>\n",
       "                     Distance: 0.73866\n",
       "            \n",
       "                     </h4>\n",
       "            <h4 style=\"font-size: 9px; padding-left: 10px; padding-right: 10px; width: 95%; word-wrap: break-word; white-space: normal;\">../data/raw/images/eUSfRwB_qY8.jpg</h4><img src=\"../data/raw/images/eUSfRwB_qY8.jpg\"/>\n",
       "            <a href=\"#!\">\n",
       "                <span class=\"ipyplot-img-close\"/>\n",
       "            </a>\n",
       "            <a href=\"#ipyplot-content-div-ZRkj89Wb9F2srHpgBgQKAH-98s2DvPaPYLPGuRQmNH252\">\n",
       "                <span class=\"ipyplot-img-expand\"/>\n",
       "            </a>\n",
       "        </div>\n",
       "    </div>\n",
       "    \n",
       "    <div class=\"ipyplot-placeholder-div-ZRkj89Wb9F2srHpgBgQKAH\">\n",
       "        <div id=\"ipyplot-content-div-ZRkj89Wb9F2srHpgBgQKAH-N52oVHd43bVaYD6rbWNPpi\" class=\"ipyplot-content-div-ZRkj89Wb9F2srHpgBgQKAH\">\n",
       "            <h4 style=\"font-size: 12px; word-wrap: break-word;\">\n",
       "                     Photo title: grayscale photo of a desert   <br/>\n",
       "                     Distance: 0.73768\n",
       "            \n",
       "                     </h4>\n",
       "            <h4 style=\"font-size: 9px; padding-left: 10px; padding-right: 10px; width: 95%; word-wrap: break-word; white-space: normal;\">../data/raw/images/h0fDDhf-MWw.jpg</h4><img src=\"../data/raw/images/h0fDDhf-MWw.jpg\"/>\n",
       "            <a href=\"#!\">\n",
       "                <span class=\"ipyplot-img-close\"/>\n",
       "            </a>\n",
       "            <a href=\"#ipyplot-content-div-ZRkj89Wb9F2srHpgBgQKAH-N52oVHd43bVaYD6rbWNPpi\">\n",
       "                <span class=\"ipyplot-img-expand\"/>\n",
       "            </a>\n",
       "        </div>\n",
       "    </div>\n",
       "    \n",
       "    <div class=\"ipyplot-placeholder-div-ZRkj89Wb9F2srHpgBgQKAH\">\n",
       "        <div id=\"ipyplot-content-div-ZRkj89Wb9F2srHpgBgQKAH-cPSWpHNp4WWaAmpMztH8Wr\" class=\"ipyplot-content-div-ZRkj89Wb9F2srHpgBgQKAH\">\n",
       "            <h4 style=\"font-size: 12px; word-wrap: break-word;\">\n",
       "                     Photo title: silhouette of person standing on desert   <br/>\n",
       "                     Distance: 0.72608\n",
       "            \n",
       "                     </h4>\n",
       "            <h4 style=\"font-size: 9px; padding-left: 10px; padding-right: 10px; width: 95%; word-wrap: break-word; white-space: normal;\">../data/raw/images/p8BM14LpLF0.jpg</h4><img src=\"../data/raw/images/p8BM14LpLF0.jpg\"/>\n",
       "            <a href=\"#!\">\n",
       "                <span class=\"ipyplot-img-close\"/>\n",
       "            </a>\n",
       "            <a href=\"#ipyplot-content-div-ZRkj89Wb9F2srHpgBgQKAH-cPSWpHNp4WWaAmpMztH8Wr\">\n",
       "                <span class=\"ipyplot-img-expand\"/>\n",
       "            </a>\n",
       "        </div>\n",
       "    </div>\n",
       "    </div>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "find_results( \"image of a man in a desert\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b35efe1b-c973-432e-9fa3-47777e9ed6ed",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "481c4649-02bb-4b75-80e1-b2609fca3371",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "id": "8195f668-ea93-47eb-8ef8-c32bd8634d9d",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-25T23:14:13.240046Z",
     "iopub.status.busy": "2023-04-25T23:14:13.239550Z",
     "iopub.status.idle": "2023-04-25T23:14:13.245733Z",
     "shell.execute_reply": "2023-04-25T23:14:13.244639Z",
     "shell.execute_reply.started": "2023-04-25T23:14:13.240012Z"
    }
   },
   "outputs": [],
   "source": [
    "sample_queries = [\n",
    "\n",
    "\"person on top of mountain\"\n",
    ", \"picture of a man in a desert\"\n",
    ", \"person in a desert\"    \n",
    "\n",
    ", \"the boy and girl on a beach\"\n",
    ", \"children in beach\"    \n",
    "\n",
    ", \"Two dogs playing in the snow\"\n",
    "\n",
    ", \"light at the end of the tunnel\"\n",
    ", \"seven wonders of the world\"\n",
    "    \n",
    "    \n",
    ", \"water droplets on a leaf\"\n",
    "    \n",
    ", \"ripley's aquarium of canada, toronto, canada\"\n",
    ", \"the butterfly atrium at hershey gardens\"\n",
    "    \n",
    ", \"salar de uyuni uyuni bolivia\"\n",
    ", \"沙漠青蛙 沙漠青蛙\" #(desert frog)\n",
    ", \"por do sol no mar\"\n",
    ", \"conhece te a ti mesmo\" #\t ( Greek for know thyself)\n",
    "\n",
    "\n",
    ", \"there is no planet b\"\n",
    "\n",
    "\n",
    ", \"nova scotia duck tolling retriever\"\n",
    "    \n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "id": "d7a2555c-3bcd-4b4e-a8c4-5021c11ae3a7",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-25T23:14:13.789049Z",
     "iopub.status.busy": "2023-04-25T23:14:13.788302Z",
     "iopub.status.idle": "2023-04-25T23:14:13.920818Z",
     "shell.execute_reply": "2023-04-25T23:14:13.919316Z",
     "shell.execute_reply.started": "2023-04-25T23:14:13.789015Z"
    }
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "96d2f2141db94a1696892a680f43fd3a",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "interactive(children=(Dropdown(description='query', options=('person on top of mountain', 'picture of a man in…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "@interact\n",
    "def interact_find_results(query=sample_queries, k =5):\n",
    "    find_results( query, k =k)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8e48da9a-4d20-4481-ba60-d3488bf4cdf0",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "46006e20-1395-4660-a2bb-653d1013cd78",
   "metadata": {},
   "source": [
    "1) How the `person on top of mountain` compare to the BM25 approach? Is it better or about the same?\n",
    "\n",
    "2) How do the results of `picture of a man in a desert` and `person in a desert` compare ? \n",
    "\n",
    "3) How do `the boy and girl on a beach` and `children in beach` compare ?\n",
    "\n",
    "4) How do the top result in `Two dogs playing in the snow` compare ?\n",
    "5) How do the results of in `ight at the end of the tunnel` look ?\n",
    "\n",
    "6) Are the results of `seven wonders of the world`relevnat ?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6ca2fb2e-adaa-40b0-a6f9-92c65bd4703a",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1fe75615-12e5-44b2-9155-6fc7e37f6221",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "adeb699a-1a4a-4011-b579-99929d4b2c71",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "a1c62e90-816d-4770-909b-54e447ee1dcc",
   "metadata": {
    "tags": []
   },
   "source": [
    "# Performance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "id": "03db9e5d-94c2-4385-91ff-d3e309baa968",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-04-25T23:19:05.319871Z",
     "iopub.status.busy": "2023-04-25T23:19:05.319069Z",
     "iopub.status.idle": "2023-04-25T23:19:21.100314Z",
     "shell.execute_reply": "2023-04-25T23:19:21.098971Z",
     "shell.execute_reply.started": "2023-04-25T23:19:05.319836Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "19.6 ms ± 587 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)\n"
     ]
    }
   ],
   "source": [
    "%%timeit\n",
    "search_query = \"Two dogs playing in the snow\"\n",
    "k=5\n",
    "text_features = model.encode(search_query, normalize_embeddings=True)\n",
    "doc_scores = text_features @ corpus.T\n",
    "\n",
    "top_items = doc_scores.argsort()[-k:][::-1]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9f93d96f-5cd3-417b-8b06-7765ae279ac0",
   "metadata": {},
   "source": [
    "In a BM-25 approach, we were able to get the results in ~3 ms"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "afc057c3-b62f-41c6-a3dc-0ef92227932b",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "environment": {
   "kernel": "python3",
   "name": "pytorch-gpu.1-13.m107",
   "type": "gcloud",
   "uri": "gcr.io/deeplearning-platform-release/pytorch-gpu.1-13:m107"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}