examples/notebooks/04-zero-shot-cold-start.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "bea8b7ed",
   "metadata": {},
   "source": [
    "# Use SetFit for Active Learning with Small-Text\n",
    "\n",
    "*Notebook 4*  \n",
    "In this tutorial we demonstrate how to use cold start initialization with [SetFit](https://github.com/huggingface/setfit), i.e. we start active learning directly without having to provide an initial set of labels.\n",
    "\n",
    "----\n",
    "\n",
    "## Overview\n",
    "\n",
    "- [Part I: Introduction and Installation](#nb4-part1-introduction)\n",
    "- [Part II: Loading, Investigating, and Preparing the AG's News Dataset](#nb4-part2-data)\n",
    "- [Part III: Active Learning](#nb4-part3-active-learning)\n",
    "- [Part IV: Plotting the results](#nb4-part4-evaluation)\n",
    "\n",
    "----\n",
    "\n",
    "<a id=\"nb4-part1-introduction\"></a>\n",
    "## I. Introduction\n",
    "\n",
    "By *SetFit* we refer to the method by Tunstall et al. from the paper [Efficient Few-Shot Learning Without Prompts](https://arxiv.org/pdf/2209.11055.pdf). SetFit is an efficient approach that is based on constrative fine-tuning on sentence-transformer embeddings. In small-text, the [Hugging Face SetFit implementation](https://github.com/huggingface/setfit) is wrapped so that you can you use with all existing small-text components.\n",
    "\n",
    "### Installation and Preparation\n",
    "\n",
    "The small-text wrapper for SetFit is a part of the transformers integration, so the installation procedure is the same as for every other example that shows small-text with transformers. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "837556c3",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[33mDEPRECATION: git+https://github.com/webis-de/small-text#egg=small-text[transformers] contains an egg fragment with a non-PEP 508 name pip 25.0 will enforce this behaviour change. A possible replacement is to use the req @ url syntax, and remove the egg fragment. Discussion can be found at https://github.com/pypa/pip/issues/11617\u001b[0m\u001b[33m\n",
      "\u001b[0mCollecting small-text[transformers]\n",
      "  Cloning https://github.com/webis-de/small-text to /tmp/pip-install-gvuaj45u/small-text_0855a21a667d4063acffef438dadbc44\n",
      "  Running command git clone --filter=blob:none --quiet https://github.com/webis-de/small-text /tmp/pip-install-gvuaj45u/small-text_0855a21a667d4063acffef438dadbc44\n",
      "  Resolved https://github.com/webis-de/small-text to commit 2aad698d90013b3305f18f56e3ebbf5959047e0d\n",
      "  Installing build dependencies ... \u001b[?25ldone\n",
      "\u001b[?25h  Getting requirements to build wheel ... \u001b[?25ldone\n",
      "\u001b[?25h  Preparing metadata (pyproject.toml) ... \u001b[?25ldone\n",
      "\u001b[?25hRequirement already satisfied: packaging in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from small-text[transformers]) (24.1)\n",
      "Requirement already satisfied: dill>=0.3.7 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from small-text[transformers]) (0.3.8)\n",
      "Requirement already satisfied: tokenizers>=0.11.5 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from small-text[transformers]) (0.19.1)\n",
      "Requirement already satisfied: scipy in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from small-text[transformers]) (1.13.1)\n",
      "Requirement already satisfied: scikit-learn>=0.24.1 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from small-text[transformers]) (1.5.0)\n",
      "Requirement already satisfied: numpy>=1.21.0 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from small-text[transformers]) (1.26.4)\n",
      "Requirement already satisfied: tqdm in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from small-text[transformers]) (4.66.4)\n",
      "Requirement already satisfied: torch>=1.6.0 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from small-text[transformers]) (2.3.0)\n",
      "Requirement already satisfied: transformers>=4.0.0 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from small-text[transformers]) (4.44.0)\n",
      "Requirement already satisfied: joblib>=1.2.0 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from scikit-learn>=0.24.1->small-text[transformers]) (1.4.2)\n",
      "Requirement already satisfied: threadpoolctl>=3.1.0 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from scikit-learn>=0.24.1->small-text[transformers]) (3.5.0)\n",
      "Requirement already satisfied: huggingface-hub<1.0,>=0.16.4 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from tokenizers>=0.11.5->small-text[transformers]) (0.24.5)\n",
      "Requirement already satisfied: filelock in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from torch>=1.6.0->small-text[transformers]) (3.14.0)\n",
      "Requirement already satisfied: networkx in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from torch>=1.6.0->small-text[transformers]) (3.2.1)\n",
      "Requirement already satisfied: nvidia-cuda-cupti-cu12==12.1.105 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from torch>=1.6.0->small-text[transformers]) (12.1.105)\n",
      "Requirement already satisfied: nvidia-cudnn-cu12==8.9.2.26 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from torch>=1.6.0->small-text[transformers]) (8.9.2.26)\n",
      "Requirement already satisfied: typing-extensions>=4.8.0 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from torch>=1.6.0->small-text[transformers]) (4.11.0)\n",
      "Requirement already satisfied: nvidia-cuda-runtime-cu12==12.1.105 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from torch>=1.6.0->small-text[transformers]) (12.1.105)\n",
      "Requirement already satisfied: nvidia-nvtx-cu12==12.1.105 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from torch>=1.6.0->small-text[transformers]) (12.1.105)\n",
      "Requirement already satisfied: fsspec in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from torch>=1.6.0->small-text[transformers]) (2024.3.1)\n",
      "Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.1.105 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from torch>=1.6.0->small-text[transformers]) (12.1.105)\n",
      "Requirement already satisfied: nvidia-curand-cu12==10.3.2.106 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from torch>=1.6.0->small-text[transformers]) (10.3.2.106)\n",
      "Requirement already satisfied: nvidia-cusolver-cu12==11.4.5.107 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from torch>=1.6.0->small-text[transformers]) (11.4.5.107)\n",
      "Requirement already satisfied: triton==2.3.0 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from torch>=1.6.0->small-text[transformers]) (2.3.0)\n",
      "Requirement already satisfied: sympy in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from torch>=1.6.0->small-text[transformers]) (1.12)\n",
      "Requirement already satisfied: nvidia-nccl-cu12==2.20.5 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from torch>=1.6.0->small-text[transformers]) (2.20.5)\n",
      "Requirement already satisfied: nvidia-cublas-cu12==12.1.3.1 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from torch>=1.6.0->small-text[transformers]) (12.1.3.1)\n",
      "Requirement already satisfied: nvidia-cusparse-cu12==12.1.0.106 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from torch>=1.6.0->small-text[transformers]) (12.1.0.106)\n",
      "Requirement already satisfied: jinja2 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from torch>=1.6.0->small-text[transformers]) (3.1.4)\n",
      "Requirement already satisfied: nvidia-cufft-cu12==11.0.2.54 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from torch>=1.6.0->small-text[transformers]) (11.0.2.54)\n",
      "Requirement already satisfied: nvidia-nvjitlink-cu12 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from nvidia-cusolver-cu12==11.4.5.107->torch>=1.6.0->small-text[transformers]) (12.5.40)\n",
      "Requirement already satisfied: pyyaml>=5.1 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from transformers>=4.0.0->small-text[transformers]) (6.0.1)\n",
      "Requirement already satisfied: regex!=2019.12.17 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from transformers>=4.0.0->small-text[transformers]) (2024.5.15)\n",
      "Requirement already satisfied: safetensors>=0.4.1 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from transformers>=4.0.0->small-text[transformers]) (0.4.3)\n",
      "Requirement already satisfied: requests in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from transformers>=4.0.0->small-text[transformers]) (2.32.1)\n",
      "Requirement already satisfied: MarkupSafe>=2.0 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from jinja2->torch>=1.6.0->small-text[transformers]) (2.1.5)\n",
      "Requirement already satisfied: charset-normalizer<4,>=2 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from requests->transformers>=4.0.0->small-text[transformers]) (3.3.2)\n",
      "Requirement already satisfied: idna<4,>=2.5 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from requests->transformers>=4.0.0->small-text[transformers]) (3.7)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from requests->transformers>=4.0.0->small-text[transformers]) (2024.2.2)\n",
      "Requirement already satisfied: urllib3<3,>=1.21.1 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from requests->transformers>=4.0.0->small-text[transformers]) (2.2.1)\n",
      "Requirement already satisfied: mpmath>=0.19 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from sympy->torch>=1.6.0->small-text[transformers]) (1.3.0)\n",
      "\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.0.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m24.3.1\u001b[0m\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpython3.9 -m pip install --upgrade pip\u001b[0m\n",
      "Note: you may need to restart the kernel to use updated packages.\n",
      "Requirement already satisfied: setfit==1.1.0 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (1.1.0)\n",
      "Requirement already satisfied: evaluate>=0.3.0 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from setfit==1.1.0) (0.4.2)\n",
      "Requirement already satisfied: sentence-transformers[train]>=3 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from setfit==1.1.0) (3.0.1)\n",
      "Requirement already satisfied: scikit-learn in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from setfit==1.1.0) (1.5.0)\n",
      "Requirement already satisfied: datasets>=2.15.0 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from setfit==1.1.0) (2.19.1)\n",
      "Requirement already satisfied: huggingface-hub>=0.23.0 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from setfit==1.1.0) (0.24.5)\n",
      "Requirement already satisfied: packaging in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from setfit==1.1.0) (24.1)\n",
      "Requirement already satisfied: transformers>=4.41.0 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from setfit==1.1.0) (4.44.0)\n",
      "Requirement already satisfied: requests>=2.19.0 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from datasets>=2.15.0->setfit==1.1.0) (2.32.1)\n",
      "Requirement already satisfied: fsspec[http]<=2024.3.1,>=2023.1.0 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from datasets>=2.15.0->setfit==1.1.0) (2024.3.1)\n",
      "Requirement already satisfied: numpy>=1.17 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from datasets>=2.15.0->setfit==1.1.0) (1.26.4)\n",
      "Requirement already satisfied: pyarrow-hotfix in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from datasets>=2.15.0->setfit==1.1.0) (0.6)\n",
      "Requirement already satisfied: filelock in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from datasets>=2.15.0->setfit==1.1.0) (3.14.0)\n",
      "Requirement already satisfied: pyyaml>=5.1 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from datasets>=2.15.0->setfit==1.1.0) (6.0.1)\n",
      "Requirement already satisfied: dill<0.3.9,>=0.3.0 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from datasets>=2.15.0->setfit==1.1.0) (0.3.8)\n",
      "Requirement already satisfied: aiohttp in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from datasets>=2.15.0->setfit==1.1.0) (3.9.5)\n",
      "Requirement already satisfied: tqdm>=4.62.1 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from datasets>=2.15.0->setfit==1.1.0) (4.66.4)\n",
      "Requirement already satisfied: pandas in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from datasets>=2.15.0->setfit==1.1.0) (2.2.2)\n",
      "Requirement already satisfied: pyarrow>=12.0.0 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from datasets>=2.15.0->setfit==1.1.0) (16.1.0)\n",
      "Requirement already satisfied: xxhash in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from datasets>=2.15.0->setfit==1.1.0) (3.4.1)\n",
      "Requirement already satisfied: multiprocess in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from datasets>=2.15.0->setfit==1.1.0) (0.70.16)\n",
      "Requirement already satisfied: typing-extensions>=3.7.4.3 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from huggingface-hub>=0.23.0->setfit==1.1.0) (4.11.0)\n",
      "Requirement already satisfied: scipy in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from sentence-transformers[train]>=3->setfit==1.1.0) (1.13.1)\n",
      "Requirement already satisfied: Pillow in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from sentence-transformers[train]>=3->setfit==1.1.0) (10.4.0)\n",
      "Requirement already satisfied: torch>=1.11.0 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from sentence-transformers[train]>=3->setfit==1.1.0) (2.3.0)\n",
      "Requirement already satisfied: accelerate>=0.20.3 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from sentence-transformers[train]>=3->setfit==1.1.0) (0.33.0)\n",
      "Requirement already satisfied: safetensors>=0.4.1 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from transformers>=4.41.0->setfit==1.1.0) (0.4.3)\n",
      "Requirement already satisfied: tokenizers<0.20,>=0.19 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from transformers>=4.41.0->setfit==1.1.0) (0.19.1)\n",
      "Requirement already satisfied: regex!=2019.12.17 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from transformers>=4.41.0->setfit==1.1.0) (2024.5.15)\n",
      "Requirement already satisfied: joblib>=1.2.0 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from scikit-learn->setfit==1.1.0) (1.4.2)\n",
      "Requirement already satisfied: threadpoolctl>=3.1.0 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from scikit-learn->setfit==1.1.0) (3.5.0)\n",
      "Requirement already satisfied: psutil in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from accelerate>=0.20.3->sentence-transformers[train]>=3->setfit==1.1.0) (5.9.8)\n",
      "Requirement already satisfied: multidict<7.0,>=4.5 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from aiohttp->datasets>=2.15.0->setfit==1.1.0) (6.0.5)\n",
      "Requirement already satisfied: yarl<2.0,>=1.0 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from aiohttp->datasets>=2.15.0->setfit==1.1.0) (1.9.4)\n",
      "Requirement already satisfied: frozenlist>=1.1.1 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from aiohttp->datasets>=2.15.0->setfit==1.1.0) (1.4.1)\n",
      "Requirement already satisfied: aiosignal>=1.1.2 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from aiohttp->datasets>=2.15.0->setfit==1.1.0) (1.3.1)\n",
      "Requirement already satisfied: attrs>=17.3.0 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from aiohttp->datasets>=2.15.0->setfit==1.1.0) (23.2.0)\n",
      "Requirement already satisfied: async-timeout<5.0,>=4.0 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from aiohttp->datasets>=2.15.0->setfit==1.1.0) (4.0.3)\n",
      "Requirement already satisfied: idna<4,>=2.5 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from requests>=2.19.0->datasets>=2.15.0->setfit==1.1.0) (3.7)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from requests>=2.19.0->datasets>=2.15.0->setfit==1.1.0) (2024.2.2)\n",
      "Requirement already satisfied: charset-normalizer<4,>=2 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from requests>=2.19.0->datasets>=2.15.0->setfit==1.1.0) (3.3.2)\n",
      "Requirement already satisfied: urllib3<3,>=1.21.1 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from requests>=2.19.0->datasets>=2.15.0->setfit==1.1.0) (2.2.1)\n",
      "Requirement already satisfied: nvidia-curand-cu12==10.3.2.106 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from torch>=1.11.0->sentence-transformers[train]>=3->setfit==1.1.0) (10.3.2.106)\n",
      "Requirement already satisfied: triton==2.3.0 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from torch>=1.11.0->sentence-transformers[train]>=3->setfit==1.1.0) (2.3.0)\n",
      "Requirement already satisfied: nvidia-cuda-cupti-cu12==12.1.105 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from torch>=1.11.0->sentence-transformers[train]>=3->setfit==1.1.0) (12.1.105)\n",
      "Requirement already satisfied: nvidia-cublas-cu12==12.1.3.1 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from torch>=1.11.0->sentence-transformers[train]>=3->setfit==1.1.0) (12.1.3.1)\n",
      "Requirement already satisfied: nvidia-nvtx-cu12==12.1.105 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from torch>=1.11.0->sentence-transformers[train]>=3->setfit==1.1.0) (12.1.105)\n",
      "Requirement already satisfied: nvidia-cudnn-cu12==8.9.2.26 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from torch>=1.11.0->sentence-transformers[train]>=3->setfit==1.1.0) (8.9.2.26)\n",
      "Requirement already satisfied: nvidia-cuda-runtime-cu12==12.1.105 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from torch>=1.11.0->sentence-transformers[train]>=3->setfit==1.1.0) (12.1.105)\n",
      "Requirement already satisfied: nvidia-cufft-cu12==11.0.2.54 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from torch>=1.11.0->sentence-transformers[train]>=3->setfit==1.1.0) (11.0.2.54)\n",
      "Requirement already satisfied: nvidia-cuda-nvrtc-cu12==12.1.105 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from torch>=1.11.0->sentence-transformers[train]>=3->setfit==1.1.0) (12.1.105)\n",
      "Requirement already satisfied: nvidia-cusolver-cu12==11.4.5.107 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from torch>=1.11.0->sentence-transformers[train]>=3->setfit==1.1.0) (11.4.5.107)\n",
      "Requirement already satisfied: sympy in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from torch>=1.11.0->sentence-transformers[train]>=3->setfit==1.1.0) (1.12)\n",
      "Requirement already satisfied: nvidia-cusparse-cu12==12.1.0.106 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from torch>=1.11.0->sentence-transformers[train]>=3->setfit==1.1.0) (12.1.0.106)\n",
      "Requirement already satisfied: networkx in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from torch>=1.11.0->sentence-transformers[train]>=3->setfit==1.1.0) (3.2.1)\n",
      "Requirement already satisfied: nvidia-nccl-cu12==2.20.5 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from torch>=1.11.0->sentence-transformers[train]>=3->setfit==1.1.0) (2.20.5)\n",
      "Requirement already satisfied: jinja2 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from torch>=1.11.0->sentence-transformers[train]>=3->setfit==1.1.0) (3.1.4)\n",
      "Requirement already satisfied: nvidia-nvjitlink-cu12 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from nvidia-cusolver-cu12==11.4.5.107->torch>=1.11.0->sentence-transformers[train]>=3->setfit==1.1.0) (12.5.40)\n",
      "Requirement already satisfied: tzdata>=2022.7 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from pandas->datasets>=2.15.0->setfit==1.1.0) (2024.1)\n",
      "Requirement already satisfied: python-dateutil>=2.8.2 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from pandas->datasets>=2.15.0->setfit==1.1.0) (2.9.0.post0)\n",
      "Requirement already satisfied: pytz>=2020.1 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from pandas->datasets>=2.15.0->setfit==1.1.0) (2024.1)\n",
      "Requirement already satisfied: six>=1.5 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from python-dateutil>=2.8.2->pandas->datasets>=2.15.0->setfit==1.1.0) (1.16.0)\n",
      "Requirement already satisfied: MarkupSafe>=2.0 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from jinja2->torch>=1.11.0->sentence-transformers[train]>=3->setfit==1.1.0) (2.1.5)\n",
      "Requirement already satisfied: mpmath>=0.19 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from sympy->torch>=1.11.0->sentence-transformers[train]>=3->setfit==1.1.0) (1.3.0)\n",
      "\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.0.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m24.3.1\u001b[0m\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpython3.9 -m pip install --upgrade pip\u001b[0m\n",
      "Note: you may need to restart the kernel to use updated packages.\n",
      "Requirement already satisfied: datasets in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (2.19.1)\n",
      "Requirement already satisfied: pyarrow-hotfix in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from datasets) (0.6)\n",
      "Requirement already satisfied: requests>=2.19.0 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from datasets) (2.32.1)\n",
      "Requirement already satisfied: packaging in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from datasets) (24.1)\n",
      "Requirement already satisfied: huggingface-hub>=0.21.2 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from datasets) (0.24.5)\n",
      "Requirement already satisfied: numpy>=1.17 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from datasets) (1.26.4)\n",
      "Requirement already satisfied: dill<0.3.9,>=0.3.0 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from datasets) (0.3.8)\n",
      "Requirement already satisfied: pyyaml>=5.1 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from datasets) (6.0.1)\n",
      "Requirement already satisfied: multiprocess in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from datasets) (0.70.16)\n",
      "Requirement already satisfied: xxhash in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from datasets) (3.4.1)\n",
      "Requirement already satisfied: fsspec[http]<=2024.3.1,>=2023.1.0 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from datasets) (2024.3.1)\n",
      "Requirement already satisfied: filelock in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from datasets) (3.14.0)\n",
      "Requirement already satisfied: pandas in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from datasets) (2.2.2)\n",
      "Requirement already satisfied: tqdm>=4.62.1 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from datasets) (4.66.4)\n",
      "Requirement already satisfied: pyarrow>=12.0.0 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from datasets) (16.1.0)\n",
      "Requirement already satisfied: aiohttp in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from datasets) (3.9.5)\n",
      "Requirement already satisfied: attrs>=17.3.0 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from aiohttp->datasets) (23.2.0)\n",
      "Requirement already satisfied: yarl<2.0,>=1.0 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from aiohttp->datasets) (1.9.4)\n",
      "Requirement already satisfied: frozenlist>=1.1.1 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from aiohttp->datasets) (1.4.1)\n",
      "Requirement already satisfied: aiosignal>=1.1.2 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from aiohttp->datasets) (1.3.1)\n",
      "Requirement already satisfied: multidict<7.0,>=4.5 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from aiohttp->datasets) (6.0.5)\n",
      "Requirement already satisfied: async-timeout<5.0,>=4.0 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from aiohttp->datasets) (4.0.3)\n",
      "Requirement already satisfied: typing-extensions>=3.7.4.3 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from huggingface-hub>=0.21.2->datasets) (4.11.0)\n",
      "Requirement already satisfied: idna<4,>=2.5 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from requests>=2.19.0->datasets) (3.7)\n",
      "Requirement already satisfied: urllib3<3,>=1.21.1 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from requests>=2.19.0->datasets) (2.2.1)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from requests>=2.19.0->datasets) (2024.2.2)\n",
      "Requirement already satisfied: charset-normalizer<4,>=2 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from requests>=2.19.0->datasets) (3.3.2)\n",
      "Requirement already satisfied: pytz>=2020.1 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from pandas->datasets) (2024.1)\n",
      "Requirement already satisfied: python-dateutil>=2.8.2 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from pandas->datasets) (2.9.0.post0)\n",
      "Requirement already satisfied: tzdata>=2022.7 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from pandas->datasets) (2024.1)\n",
      "Requirement already satisfied: six>=1.5 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from python-dateutil>=2.8.2->pandas->datasets) (1.16.0)\n",
      "\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.0.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m24.3.1\u001b[0m\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpython3.9 -m pip install --upgrade pip\u001b[0m\n",
      "Note: you may need to restart the kernel to use updated packages.\n",
      "Requirement already satisfied: matplotlib in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (3.9.0)\n",
      "Requirement already satisfied: pyparsing>=2.3.1 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from matplotlib) (3.1.2)\n",
      "Requirement already satisfied: contourpy>=1.0.1 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from matplotlib) (1.2.1)\n",
      "Requirement already satisfied: cycler>=0.10 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from matplotlib) (0.12.1)\n",
      "Requirement already satisfied: packaging>=20.0 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from matplotlib) (24.1)\n",
      "Requirement already satisfied: kiwisolver>=1.3.1 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from matplotlib) (1.4.5)\n",
      "Requirement already satisfied: fonttools>=4.22.0 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from matplotlib) (4.53.1)\n",
      "Requirement already satisfied: numpy>=1.23 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from matplotlib) (1.26.4)\n",
      "Requirement already satisfied: importlib-resources>=3.2.0 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from matplotlib) (6.4.2)\n",
      "Requirement already satisfied: pillow>=8 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from matplotlib) (10.4.0)\n",
      "Requirement already satisfied: python-dateutil>=2.7 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from matplotlib) (2.9.0.post0)\n",
      "Requirement already satisfied: zipp>=3.1.0 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from importlib-resources>=3.2.0->matplotlib) (3.20.0)\n",
      "Requirement already satisfied: six>=1.5 in /home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages (from python-dateutil>=2.7->matplotlib) (1.16.0)\n",
      "\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m A new release of pip is available: \u001b[0m\u001b[31;49m23.0.1\u001b[0m\u001b[39;49m -> \u001b[0m\u001b[32;49m24.3.1\u001b[0m\n",
      "\u001b[1m[\u001b[0m\u001b[34;49mnotice\u001b[0m\u001b[1;39;49m]\u001b[0m\u001b[39;49m To update, run: \u001b[0m\u001b[32;49mpython3.9 -m pip install --upgrade pip\u001b[0m\n",
      "Note: you may need to restart the kernel to use updated packages.\n",
      "env: TOKENIZERS_PARALLELISM=false\n"
     ]
    }
   ],
   "source": [
    "%pip install small-text[transformers]==2.0.0dev1\n",
    "\n",
    "# setfit is an optional dependency and must be installed as well\n",
    "%pip install setfit==1.1.0\n",
    "%pip install datasets\n",
    "%pip install matplotlib\n",
    "\n",
    "%env TOKENIZERS_PARALLELISM=false\n",
    "\n",
    "import logging\n",
    "import warnings\n",
    "\n",
    "import datasets\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "plt.rc('figure', titlesize=22)\n",
    "plt.rc('axes', titlesize=22, labelsize=20, linewidth=1.2)\n",
    "plt.rc('xtick', labelsize=14)\n",
    "plt.rc('ytick', labelsize=14)\n",
    "plt.rc('legend', fontsize=16)\n",
    "plt.rc('lines', linewidth=2)\n",
    "\n",
    "datasets.logging.set_verbosity_error()\n",
    "\n",
    "# disables the progress bar for notebooks: https://github.com/huggingface/datasets/issues/2651\n",
    "datasets.logging.get_verbosity = lambda: logging.NOTSET\n",
    "\n",
    "for logger_name in ['setfit.modeling', 'setfit.trainer', 'sentence_transformers']:\n",
    "    logger = logging.getLogger(logger_name)\n",
    "    logger.setLevel(logging.ERROR)\n",
    "\n",
    "warnings.filterwarnings(action='ignore', category=FutureWarning, message=r'.*`clean_up_tokenization_spaces` was not set.*')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bdf5fd19",
   "metadata": {},
   "source": [
    "----\n",
    "\n",
    "<a id=\"nb4-part2-data\"></a>\n",
    "## II. Loading the Data\n",
    "\n",
    "We use the \"AG's News\" dataset for this demonstration. It consists of news articles which are categorized into four mutually exclusive classes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "754e5b56",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "First 10 training samples:\n",
      "\n",
      "2   Wall St. Bears Claw Back Into the Black (Reuters) Reuters - Short-sellers, Wall Street's dwindling\\band of ultra-cynics, are seeing green again.\n",
      "2   Carlyle Looks Toward Commercial Aerospace (Reuters) Reuters - Private investment firm Carlyle Group,\\which has a reputation for making well-timed and occasionally\\controversial plays in the defense industry, has quietly placed\\its bets on another part of the market.\n",
      "2   Oil and Economy Cloud Stocks' Outlook (Reuters) Reuters - Soaring crude prices plus worries\\about the economy and the outlook for earnings are expected to\\hang over the stock market next week during the depth of the\\summer doldrums.\n",
      "2   Iraq Halts Oil Exports from Main Southern Pipeline (Reuters) Reuters - Authorities have halted oil export\\flows from the main pipeline in southern Iraq after\\intelligence showed a rebel militia could strike\\infrastructure, an oil official said on Saturday.\n",
      "2   Oil prices soar to all-time record, posing new menace to US economy (AFP) AFP - Tearaway world oil prices, toppling records and straining wallets, present a new economic menace barely three months before the US presidential elections.\n",
      "2   Stocks End Up, But Near Year Lows (Reuters) Reuters - Stocks ended slightly higher on Friday\\but stayed near lows for the year as oil prices surged past  #36;46\\a barrel, offsetting a positive outlook from computer maker\\Dell Inc. (DELL.O)\n",
      "2   Money Funds Fell in Latest Week (AP) AP - Assets of the nation's retail money market mutual funds fell by  #36;1.17 billion in the latest week to  #36;849.98 trillion, the Investment Company Institute said Thursday.\n",
      "2   Fed minutes show dissent over inflation (USATODAY.com) USATODAY.com - Retail sales bounced back a bit in July, and new claims for jobless benefits fell last week, the government said Thursday, indicating the economy is improving from a midsummer slump.\n",
      "2   Safety Net (Forbes.com) Forbes.com - After earning a PH.D. in Sociology, Danny Bazil Riley started to work as the general manager at a commercial real estate firm at an annual base salary of  #36;70,000. Soon after, a financial planner stopped by his desk to drop off brochures about insurance benefits available through his employer. But, at 32, \"buying insurance was the furthest thing from my mind,\" says Riley.\n",
      "2   Wall St. Bears Claw Back Into the Black  NEW YORK (Reuters) - Short-sellers, Wall Street's dwindling  band of ultra-cynics, are seeing green again.\n"
     ]
    }
   ],
   "source": [
    "import datasets\n",
    "import numpy as np\n",
    "\n",
    "\n",
    "# disables the progress bar for notebooks: https://github.com/huggingface/datasets/issues/2651\n",
    "datasets.logging.get_verbosity = lambda: logging.NOTSET\n",
    "\n",
    "raw_dataset = datasets.load_dataset('ag_news')\n",
    "num_classes = np.unique(raw_dataset['train']['label']).shape[0]\n",
    "\n",
    "print('First 10 training samples:\\n')\n",
    "for i in range(10):\n",
    "    print(raw_dataset['train']['label'][i], ' ', raw_dataset['train']['text'][i])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "80d66199",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ClassLabel(names=['World', 'Sports', 'Business', 'Sci/Tech'], id=None)\n"
     ]
    }
   ],
   "source": [
    "print(raw_dataset['train'].features['label'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1742f7d7",
   "metadata": {},
   "source": [
    "### Preparing the Data\n",
    "\n",
    "The SetFit Classifier uses the `TextData` structure as input. Such datasets can be easily created from raw text and labels using `TextDataset.from_arrays()`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "be151ce8",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from small_text import TextDataset\n",
    "\n",
    "num_classes = raw_dataset['train'].features['label'].num_classes\n",
    "\n",
    "target_labels = np.arange(num_classes)\n",
    "\n",
    "train = TextDataset.from_arrays(raw_dataset['train']['text'], np.array(raw_dataset['train']['label']), target_labels=target_labels)\n",
    "test = TextDataset.from_arrays(raw_dataset['test']['text'], np.array(raw_dataset['test']['label']), target_labels=target_labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3db26811",
   "metadata": {},
   "source": [
    "----\n",
    "\n",
    "<a id=\"nb4-part3-active-learning\"></a>\n",
    "## III. Active Learning\n",
    "\n",
    "First, we setup the classifier. Compared to the previous notebook examples, we use the SetFit factory and model arguments instead of the transformer-based variants."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "2184e4b7",
   "metadata": {},
   "outputs": [],
   "source": [
    "# TODO: use short imports\n",
    "from small_text.integrations.transformers.classifiers.setfit import SetFitModelArguments\n",
    "from small_text.integrations.transformers.classifiers.factories import SetFitClassificationFactory\n",
    "\n",
    "sentence_transformer_model_name = 'sentence-transformers/paraphrase-mpnet-base-v2'\n",
    "setfit_model_args = SetFitModelArguments(sentence_transformer_model_name)\n",
    "\n",
    "clf_factory = SetFitClassificationFactory(setfit_model_args, num_classes)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "017ebf3d",
   "metadata": {},
   "source": [
    "## Setting up the Active Learner\n",
    "\n",
    "Here, we construct a `PoolBasedActiveLearner` instance which requires a classifier factory, a query strategy, and the train dataset.\n",
    "\n",
    "To obtain a first model, we initialize the active learner by providing the true labels for 20 sentences. This corresponds to an initial labeling the real-world setting."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "66774521",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages/small_text/integrations/transformers/classifiers/setfit.py:262: DeprecationWarning: `SetFitTrainer` has been deprecated and will be removed in v2.0.0 of SetFit. Please use `Trainer` instead.\n",
      "  trainer = SetFitTrainer(\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "998cd2c262be4022a599c7fbf436df4a",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/4 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages/small_text/integrations/transformers/classifiers/setfit.py:270: DeprecationWarning: `SetFitTrainer.train` does not accept keyword arguments anymore. Please provide training arguments via a `TrainingArguments` instance to the `SetFitTrainer` initialisation or the `SetFitTrainer.train` method.\n",
      "  trainer.train(max_length=self.max_seq_len, **setfit_train_kwargs)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='5' max='5' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [5/5 00:01, Epoch 1/1]\n",
       "    </div>\n",
       "    <table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       " <tr style=\"text-align: left;\">\n",
       "      <th>Step</th>\n",
       "      <th>Training Loss</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>0.040100</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table><p>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Computing widget examples:   0%|          | 0/1 [00:00<?, ?example/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[]\n"
     ]
    }
   ],
   "source": [
    "from small_text import (\n",
    "    PoolBasedActiveLearner, \n",
    "    BreakingTies,\n",
    "    SubsamplingQueryStrategy\n",
    ")\n",
    "\n",
    "# define a query strategy and initialize a pool-based active learner\n",
    "query_strategy = SubsamplingQueryStrategy(BreakingTies())\n",
    "\n",
    "def initialize_with_coldstart():\n",
    "    cold_start_labels = np.array([0, 1, 2, 3])\n",
    "    cold_start_dataset = TextDataset.from_arrays([\n",
    "            'This is about world news.',\n",
    "            'This is about sports.',\n",
    "            'This is about business.',\n",
    "            'This is about science and technology.'\n",
    "        ], \n",
    "        cold_start_labels,\n",
    "        target_labels=target_labels)\n",
    "\n",
    "    train = TextDataset.from_arrays(cold_start_dataset.x + raw_dataset['train']['text'], \n",
    "                                    np.append(cold_start_labels, np.array(raw_dataset['train']['label'])), \n",
    "                                    target_labels=target_labels)\n",
    "    \n",
    "    # suppress progress bars in jupyter notebook\n",
    "    setfit_train_kwargs = {'show_progress_bar': False}\n",
    "\n",
    "    active_learner = PoolBasedActiveLearner(clf_factory, query_strategy, train, fit_kwargs={'setfit_train_kwargs': setfit_train_kwargs})\n",
    "    \n",
    "    clf = clf_factory.new()\n",
    "    clf.fit(cold_start_dataset, setfit_train_kwargs=setfit_train_kwargs)\n",
    "\n",
    "    active_learner.initialize(clf)\n",
    "    print(active_learner.y)\n",
    "\n",
    "    return active_learner, train\n",
    "\n",
    "\n",
    "active_learner, train = initialize_with_coldstart()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e8888591",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Active Learning\n",
    "\n",
    "The main active learning loop queries the unlabeled pool and thereby decides which documents are labeled next.\n",
    "We then provide the labels for those documents and the active learner retrains the model.\n",
    "After each query, we evaluate the current model against the test set and save the result.\n",
    "\n",
    "\n",
    "Note: This is active learning as it is done in a scientific simulation. In reality, the label feedback would have been given by human annotators."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "df13d646-ebde-4ff5-8aa1-b59aa4794bac",
   "metadata": {},
   "source": [
    "#### Active Learning Loop"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "aacd866c",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[]\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages/small_text/integrations/transformers/classifiers/setfit.py:262: DeprecationWarning: `SetFitTrainer` has been deprecated and will be removed in v2.0.0 of SetFit. Please use `Trainer` instead.\n",
      "  trainer = SetFitTrainer(\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "011c644b5b994a129d12c92111354a78",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/20 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages/small_text/integrations/transformers/classifiers/setfit.py:270: DeprecationWarning: `SetFitTrainer.train` does not accept keyword arguments anymore. Please provide training arguments via a `TrainingArguments` instance to the `SetFitTrainer` initialisation or the `SetFitTrainer.train` method.\n",
      "  trainer.train(max_length=self.max_seq_len, **setfit_train_kwargs)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='25' max='25' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [25/25 00:04, Epoch 1/1]\n",
       "    </div>\n",
       "    <table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       " <tr style=\"text-align: left;\">\n",
       "      <th>Step</th>\n",
       "      <th>Training Loss</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>0.349000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table><p>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Computing widget examples:   0%|          | 0/1 [00:00<?, ?example/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "---------------\n",
      "Iteration #0 (20 samples)\n",
      "Train accuracy: 1.00\n",
      "Test accuracy: 0.53\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages/small_text/integrations/transformers/classifiers/setfit.py:262: DeprecationWarning: `SetFitTrainer` has been deprecated and will be removed in v2.0.0 of SetFit. Please use `Trainer` instead.\n",
      "  trainer = SetFitTrainer(\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "3f2a584f80cf477cb8528096d84caf2d",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/40 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages/small_text/integrations/transformers/classifiers/setfit.py:270: DeprecationWarning: `SetFitTrainer.train` does not accept keyword arguments anymore. Please provide training arguments via a `TrainingArguments` instance to the `SetFitTrainer` initialisation or the `SetFitTrainer.train` method.\n",
      "  trainer.train(max_length=self.max_seq_len, **setfit_train_kwargs)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='50' max='50' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [50/50 00:07, Epoch 1/1]\n",
       "    </div>\n",
       "    <table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       " <tr style=\"text-align: left;\">\n",
       "      <th>Step</th>\n",
       "      <th>Training Loss</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>0.390100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>50</td>\n",
       "      <td>0.106300</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table><p>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Computing widget examples:   0%|          | 0/1 [00:00<?, ?example/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "---------------\n",
      "Iteration #1 (40 samples)\n",
      "Train accuracy: 1.00\n",
      "Test accuracy: 0.82\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages/small_text/integrations/transformers/classifiers/setfit.py:262: DeprecationWarning: `SetFitTrainer` has been deprecated and will be removed in v2.0.0 of SetFit. Please use `Trainer` instead.\n",
      "  trainer = SetFitTrainer(\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "1e85d769a58a4e588f5ad9672c76b2b9",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/60 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages/small_text/integrations/transformers/classifiers/setfit.py:270: DeprecationWarning: `SetFitTrainer.train` does not accept keyword arguments anymore. Please provide training arguments via a `TrainingArguments` instance to the `SetFitTrainer` initialisation or the `SetFitTrainer.train` method.\n",
      "  trainer.train(max_length=self.max_seq_len, **setfit_train_kwargs)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='75' max='75' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [75/75 00:10, Epoch 1/1]\n",
       "    </div>\n",
       "    <table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       " <tr style=\"text-align: left;\">\n",
       "      <th>Step</th>\n",
       "      <th>Training Loss</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>0.404800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>50</td>\n",
       "      <td>0.159200</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table><p>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Computing widget examples:   0%|          | 0/1 [00:00<?, ?example/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "---------------\n",
      "Iteration #2 (60 samples)\n",
      "Train accuracy: 1.00\n",
      "Test accuracy: 0.86\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages/small_text/integrations/transformers/classifiers/setfit.py:262: DeprecationWarning: `SetFitTrainer` has been deprecated and will be removed in v2.0.0 of SetFit. Please use `Trainer` instead.\n",
      "  trainer = SetFitTrainer(\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "ee611ee9fc7d4d4da3b9daa20dcdd33b",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/80 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages/small_text/integrations/transformers/classifiers/setfit.py:270: DeprecationWarning: `SetFitTrainer.train` does not accept keyword arguments anymore. Please provide training arguments via a `TrainingArguments` instance to the `SetFitTrainer` initialisation or the `SetFitTrainer.train` method.\n",
      "  trainer.train(max_length=self.max_seq_len, **setfit_train_kwargs)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='100' max='100' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [100/100 00:13, Epoch 1/1]\n",
       "    </div>\n",
       "    <table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       " <tr style=\"text-align: left;\">\n",
       "      <th>Step</th>\n",
       "      <th>Training Loss</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>0.350000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>50</td>\n",
       "      <td>0.205900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>100</td>\n",
       "      <td>0.052600</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table><p>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Computing widget examples:   0%|          | 0/1 [00:00<?, ?example/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "---------------\n",
      "Iteration #3 (80 samples)\n",
      "Train accuracy: 1.00\n",
      "Test accuracy: 0.87\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages/small_text/integrations/transformers/classifiers/setfit.py:262: DeprecationWarning: `SetFitTrainer` has been deprecated and will be removed in v2.0.0 of SetFit. Please use `Trainer` instead.\n",
      "  trainer = SetFitTrainer(\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "f26e8d6a7c624d8e89ea9e100d9164e8",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/100 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages/small_text/integrations/transformers/classifiers/setfit.py:270: DeprecationWarning: `SetFitTrainer.train` does not accept keyword arguments anymore. Please provide training arguments via a `TrainingArguments` instance to the `SetFitTrainer` initialisation or the `SetFitTrainer.train` method.\n",
      "  trainer.train(max_length=self.max_seq_len, **setfit_train_kwargs)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='125' max='125' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [125/125 00:16, Epoch 1/1]\n",
       "    </div>\n",
       "    <table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       " <tr style=\"text-align: left;\">\n",
       "      <th>Step</th>\n",
       "      <th>Training Loss</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>0.328900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>50</td>\n",
       "      <td>0.209300</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>100</td>\n",
       "      <td>0.057600</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table><p>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Computing widget examples:   0%|          | 0/1 [00:00<?, ?example/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "---------------\n",
      "Iteration #4 (100 samples)\n",
      "Train accuracy: 1.00\n",
      "Test accuracy: 0.88\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages/small_text/integrations/transformers/classifiers/setfit.py:262: DeprecationWarning: `SetFitTrainer` has been deprecated and will be removed in v2.0.0 of SetFit. Please use `Trainer` instead.\n",
      "  trainer = SetFitTrainer(\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "5d824b5c0385406abcd7086fe1fec6fb",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/120 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages/small_text/integrations/transformers/classifiers/setfit.py:270: DeprecationWarning: `SetFitTrainer.train` does not accept keyword arguments anymore. Please provide training arguments via a `TrainingArguments` instance to the `SetFitTrainer` initialisation or the `SetFitTrainer.train` method.\n",
      "  trainer.train(max_length=self.max_seq_len, **setfit_train_kwargs)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='150' max='150' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [150/150 00:19, Epoch 1/1]\n",
       "    </div>\n",
       "    <table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       " <tr style=\"text-align: left;\">\n",
       "      <th>Step</th>\n",
       "      <th>Training Loss</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>0.359400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>50</td>\n",
       "      <td>0.251500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>100</td>\n",
       "      <td>0.116800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>150</td>\n",
       "      <td>0.031500</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table><p>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Computing widget examples:   0%|          | 0/1 [00:00<?, ?example/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "---------------\n",
      "Iteration #5 (120 samples)\n",
      "Train accuracy: 1.00\n",
      "Test accuracy: 0.88\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages/small_text/integrations/transformers/classifiers/setfit.py:262: DeprecationWarning: `SetFitTrainer` has been deprecated and will be removed in v2.0.0 of SetFit. Please use `Trainer` instead.\n",
      "  trainer = SetFitTrainer(\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "a60b5647726241d7ae6b981b7a883d84",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/140 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages/small_text/integrations/transformers/classifiers/setfit.py:270: DeprecationWarning: `SetFitTrainer.train` does not accept keyword arguments anymore. Please provide training arguments via a `TrainingArguments` instance to the `SetFitTrainer` initialisation or the `SetFitTrainer.train` method.\n",
      "  trainer.train(max_length=self.max_seq_len, **setfit_train_kwargs)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='175' max='175' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [175/175 00:22, Epoch 1/1]\n",
       "    </div>\n",
       "    <table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       " <tr style=\"text-align: left;\">\n",
       "      <th>Step</th>\n",
       "      <th>Training Loss</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>0.293800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>50</td>\n",
       "      <td>0.250800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>100</td>\n",
       "      <td>0.126500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>150</td>\n",
       "      <td>0.014200</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table><p>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Computing widget examples:   0%|          | 0/1 [00:00<?, ?example/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "---------------\n",
      "Iteration #6 (140 samples)\n",
      "Train accuracy: 1.00\n",
      "Test accuracy: 0.88\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages/small_text/integrations/transformers/classifiers/setfit.py:262: DeprecationWarning: `SetFitTrainer` has been deprecated and will be removed in v2.0.0 of SetFit. Please use `Trainer` instead.\n",
      "  trainer = SetFitTrainer(\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "2f68a6ce6807432ebe442435a405b012",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/160 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages/small_text/integrations/transformers/classifiers/setfit.py:270: DeprecationWarning: `SetFitTrainer.train` does not accept keyword arguments anymore. Please provide training arguments via a `TrainingArguments` instance to the `SetFitTrainer` initialisation or the `SetFitTrainer.train` method.\n",
      "  trainer.train(max_length=self.max_seq_len, **setfit_train_kwargs)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='200' max='200' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [200/200 00:27, Epoch 1/1]\n",
       "    </div>\n",
       "    <table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       " <tr style=\"text-align: left;\">\n",
       "      <th>Step</th>\n",
       "      <th>Training Loss</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>0.427100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>50</td>\n",
       "      <td>0.258200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>100</td>\n",
       "      <td>0.148400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>150</td>\n",
       "      <td>0.033400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>200</td>\n",
       "      <td>0.003700</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table><p>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Computing widget examples:   0%|          | 0/1 [00:00<?, ?example/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "---------------\n",
      "Iteration #7 (160 samples)\n",
      "Train accuracy: 1.00\n",
      "Test accuracy: 0.89\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages/small_text/integrations/transformers/classifiers/setfit.py:262: DeprecationWarning: `SetFitTrainer` has been deprecated and will be removed in v2.0.0 of SetFit. Please use `Trainer` instead.\n",
      "  trainer = SetFitTrainer(\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "5fbe34e1d65848dc8c2579cce93900df",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/180 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages/small_text/integrations/transformers/classifiers/setfit.py:270: DeprecationWarning: `SetFitTrainer.train` does not accept keyword arguments anymore. Please provide training arguments via a `TrainingArguments` instance to the `SetFitTrainer` initialisation or the `SetFitTrainer.train` method.\n",
      "  trainer.train(max_length=self.max_seq_len, **setfit_train_kwargs)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='225' max='225' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [225/225 00:29, Epoch 1/1]\n",
       "    </div>\n",
       "    <table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       " <tr style=\"text-align: left;\">\n",
       "      <th>Step</th>\n",
       "      <th>Training Loss</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>0.404500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>50</td>\n",
       "      <td>0.257200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>100</td>\n",
       "      <td>0.165400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>150</td>\n",
       "      <td>0.035500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>200</td>\n",
       "      <td>0.003300</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table><p>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "9dc3d594dc804500baab915769904f41",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Computing widget examples:   0%|          | 0/1 [00:00<?, ?example/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "---------------\n",
      "Iteration #8 (180 samples)\n",
      "Train accuracy: 1.00\n",
      "Test accuracy: 0.89\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages/small_text/integrations/transformers/classifiers/setfit.py:262: DeprecationWarning: `SetFitTrainer` has been deprecated and will be removed in v2.0.0 of SetFit. Please use `Trainer` instead.\n",
      "  trainer = SetFitTrainer(\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "445e1316011c4085ad7a6d0ca90f9b0f",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map:   0%|          | 0/200 [00:00<?, ? examples/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages/small_text/integrations/transformers/classifiers/setfit.py:270: DeprecationWarning: `SetFitTrainer.train` does not accept keyword arguments anymore. Please provide training arguments via a `TrainingArguments` instance to the `SetFitTrainer` initialisation or the `SetFitTrainer.train` method.\n",
      "  trainer.train(max_length=self.max_seq_len, **setfit_train_kwargs)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "    <div>\n",
       "      \n",
       "      <progress value='250' max='250' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
       "      [250/250 00:32, Epoch 1/1]\n",
       "    </div>\n",
       "    <table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       " <tr style=\"text-align: left;\">\n",
       "      <th>Step</th>\n",
       "      <th>Training Loss</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>0.321800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>50</td>\n",
       "      <td>0.273800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>100</td>\n",
       "      <td>0.180100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>150</td>\n",
       "      <td>0.051200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>200</td>\n",
       "      <td>0.004100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>250</td>\n",
       "      <td>0.002000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table><p>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Computing widget examples:   0%|          | 0/1 [00:00<?, ?example/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "---------------\n",
      "Iteration #9 (200 samples)\n",
      "Train accuracy: 1.00\n",
      "Test accuracy: 0.89\n"
     ]
    }
   ],
   "source": [
    "import gc\n",
    "import torch\n",
    "from sklearn.metrics import accuracy_score\n",
    "\n",
    "\n",
    "num_queries = 10\n",
    "\n",
    "\n",
    "def evaluate(active_learner, train, test):\n",
    "    \n",
    "    if len(train) == 0:\n",
    "        return np.nan\n",
    "    \n",
    "    y_pred = active_learner.classifier.predict(train)\n",
    "    y_pred_test = active_learner.classifier.predict(test)\n",
    "    \n",
    "    test_acc = accuracy_score(y_pred_test, test.y)\n",
    "\n",
    "    print('Train accuracy: {:.2f}'.format(accuracy_score(y_pred, train.y)))\n",
    "    print('Test accuracy: {:.2f}'.format(test_acc))\n",
    "    \n",
    "    return test_acc\n",
    "\n",
    "print(active_learner.y)\n",
    "results_setfit = []\n",
    "results_setfit.append(evaluate(active_learner, train[active_learner.indices_labeled], test))\n",
    "    \n",
    "for i in range(num_queries):\n",
    "\n",
    "    # ...where each iteration consists of labelling 20 samples\n",
    "    q_indices = active_learner.query(num_samples=20)\n",
    "\n",
    "    # Simulate user interaction here. Replace this for real-world usage.\n",
    "    y = train.y[q_indices]\n",
    "\n",
    "    # Return the labels for the current query to the active learner.\n",
    "    active_learner.update(y)\n",
    "    \n",
    "    # memory fix: https://github.com/UKPLab/sentence-transformers/issues/487, https://github.com/UKPLab/sentence-transformers/issues/1793\n",
    "    gc.collect()\n",
    "    torch.cuda.empty_cache()\n",
    "    \n",
    "    print('---------------')\n",
    "    print('Iteration #{:d} ({} samples)'.format(i, len(active_learner.indices_labeled)))\n",
    "    results_setfit.append(evaluate(active_learner, train[active_learner.indices_labeled], test))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5eeb9a63",
   "metadata": {
    "tags": []
   },
   "source": [
    "----\n",
    "\n",
    "<a id=\"nb4-part4-evaluation\"></a>\n",
    "## IV. Plotting the Results\n",
    "\n",
    "With the previous results we can plot a [learning curve](https://en.wikipedia.org/wiki/Learning_curve_(machine_learning)) to visualize the performance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "278c1d13",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<module 'matplotlib.pyplot' from '/home/cschroeder/.pyenv/versions/3.9.18/envs/notebooks/lib/python3.9/site-packages/matplotlib/pyplot.py'>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABAwAAAK6CAYAAAC9oM4kAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy80BEi2AAAACXBIWXMAAA9hAAAPYQGoP6dpAACJ+0lEQVR4nOzdeXhU5f3+8ftMJgvZIQECCYRFBIQkgIogKFKQqHUXhVJFrFqrlbpQW6FWtLYoLS6t669fWkCtFRTBXVA2RUDRQgIIsgkkECABshGSzGTO748wY4YEksAkZ2byfl3XXEyes33Gplyce57zeQzTNE0BAAAAAADUYLO6AAAAAAAA4H8IDAAAAAAAQC0EBgAAAAAAoBYCAwAAAAAAUAuBAQAAAAAAqIXAAAAAAAAA1EJgAAAAAAAAaiEwAAAAAAAAtditLqClS0pK0tGjR9W5c2erSwEAAAAAtAB79uxRVFSU9u/ff8r9mGFgsaNHj8rhcFhdBgAAAACghXA4HDp69Gi9+zHDwGLumQWbNm2yuBIAAAAAQEvQp0+fBu3HDAMAAAAAAFALgQEAAAAAAKgloAODtWvX6oorrlB8fLyioqI0aNAgzZs3r1Hn2Lx5s37+858rKSlJ4eHhSk1N1X333afDhw+f9JhFixZp2LBhiomJUWxsrIYPH64lS5ac6ccBAAAAAMBvBGwPg2XLlikzM1MREREaO3asYmJiNH/+fI0ZM0Y5OTmaNGlSvedYs2aNRo4cqWPHjumaa65R9+7dtX79ev3jH//QJ598olWrVikhIcHrmNdff1233HKL2rZtqwkTJkiS5s6dq0svvVTz5s3T6NGjm+LjAgAAAADQrAzTNE2ri2gsp9OpXr16KTc3V2vWrFG/fv0kSUVFRRo4cKB27dqlrVu3KjU19ZTnSUtL08aNG/Xuu+/q6quv9oz/7W9/0+9+9zvdddddeuWVVzzjR44cUbdu3WS327Vu3TqlpKRIknJzc9W/f39J0s6dOxUTE9Pgz+JuNkHTQwAAAABAc2jofWhAPpKwdOlS7dixQ+PGjfOEBZIUFxenKVOmqLKyUnPmzDnlOXbs2KGNGzfq/PPP9woLJGnSpElKSEjQa6+95rXUxFtvvaXCwkJNnDjRExZIUkpKiu69914VFBRowYIFvvmQAAAAAABYKCADg+XLl0uSRo0aVWtbZmamJGnFihWnPMf+/fslSV27dq21zWazqXPnziorK9OaNWt8el0AAAAAAAJBQAYG27ZtkyT16NGj1rakpCRFR0d79jmZxMRESdIPP/xQa5vL5dKePXskSVu3bm3Qdd1j9V23oqJCxcXFnpfL5VIAPhUCAAAAAAhyAdn0sKioSFL1Iwh1iY2N9exzMmeffba6deumtWvX6sMPP9RPf/pTz7bnnntOhw4dkiQVFhY26LqxsbFe+5zMk08+qccff9xrrG3btqc85mQcDoeqqqpO61gEP5vNptDQUBmGYXUpAAAAAAJQQAYGvmAYhl566SVdddVVuvrqq3Xttdeqe/fuysrK0uLFi5WWlqYNGzbIZvPtJIzJkyfrwQcf9Px8wQUXNPqGrri4WAUFBaqoqPBpbQg+ISEhioyMVLt27RQWFmZ1OQAAAAACSEAGBu5v+E/2bX5xcbFat25d73kyMzP1xRdf6IknntDSpUv14Ycfqm/fvlqwYIGWLFmiDRs2qF27dnVe98TlFouLi732OZnw8HCFh4d7fm5sIFFcXKy9e/cqOjpaiYmJfIOMOpmmqaqqKh07dkxFRUXatWuXUlJSFBkZaXVpAAAAAAJEQAYGNfsFnHvuuV7b9u/fr9LSUg0cOLBB57rgggv0wQcf1Bp/7rnnJEnnnXee13W/+eYbbdu2rVZgcKr+Br5UUFCg6OhopaSkEBSgXtHR0WrTpo12796tgoICde7c2eqSAAAAAASIgGx6OGzYMEnS4sWLa21btGiR1z6nY/fu3Vq5cqXOOeccpaWlNdt16+NwOFRRUaG4uDjCAjRYSEiI2rRpo6NHj8rpdFpdDgAAAIAAEZCBwYgRI9StWze98cYbWr9+vWe8qKhI06ZNU1hYmMaPH+8Zz8vL05YtW2o9wlBaWlprhYKioiLdcsstqqqq0pNPPum17aabblJcXJyef/555ebmesZzc3P1wgsvKDExUdddd50PP6k3d4PD0NDQJrsGgpP7MRgCAwAAAAANFZCPJNjtds2cOVOZmZm6+OKLNXbsWMXExGj+/PnavXu3ZsyYoS5dunj2nzx5subMmaNZs2ZpwoQJnvGFCxdqypQp+slPfqKOHTvq4MGDeu+995Sfn68nnnhCV199tdd1W7durRdeeEG33HKLBgwYoDFjxkiS5s6dq0OHDmnu3LmKiYlp8s/P7AI0Fr8zAAAAABorIAMDSRo+fLhWrlypqVOnau7cuXI4HEpLS9P06dM9N/L1SUtLU0ZGhhYvXqyCggLFxcVp0KBBevDBBzV8+PA6j7n55puVmJioadOmadasWTIMQ+eee64eeeQRjRw50pcfEQAAAAAAyxjmiXPy0az69OkjSdq0aVO9+5aXl+uHH35Q165dFRER0dSlIYjwuwMAAADAraH3oQHZwwAAAAAAADQtAgMEpW3btunee+/VOeeco6ioKEVERCglJUXnn3++7r33Xs2fP79Z6pg9e7YMwzjlKz4+XpK0fPlyGYahSy65pFlqAwAAAIBTCdgeBsDJvPPOOxo3bpwqKiqUkJCgIUOGqG3btjpy5IjWr1+vF198UW+++aZuuOGGM7rOrl271LVrV6WmpmrXrl2n3DcqKkqjR4+uc1tkZGS913I3LeQJIgAAAADNhcAAQeXAgQO69dZbVVFRoUmTJunPf/5zrWf2v/32W7399tvNWldiYqJmz559yn0GDhyozZs3NyhAAAAAAICmRmCAoPLBBx+otLRUHTt21IwZM+rc59xzz9W5557bzJXVLzIyUr169bK6DAAAAACQRA8DBJkDBw5Iktq2bdvoY51Op2bOnKlLLrlEbdq0UXh4uLp27aq7775bOTk5XvtOmDBBXbt2lSTt3r27Vl+C01FXD4PHHnvM63wnXqe+RyEAAAAA4HQxwwBBpXPnzpKkjRs3asmSJRoxYkSDjispKdHVV1+t5cuXKzo6Wueee67atm2rDRs26JVXXtFbb72lTz/9VP3795ckDR06VKWlpZo/f/4p+xOcqX79+unWW2/VnDlzJEm33nqr1/bo6OgmuS4AAAAAEBggqFx77bVKTk7W3r17demll2rYsGEaMWKEBgwYoPPPP/+kMw9+9atfafny5bryyiv1r3/9S+3atfNse+655/TAAw9ozJgx2rx5s0JCQnTHHXdo5MiRmj9/foP6E5zJ57n22ms9gUFTXQcAAACwkstlqso0VeU6/jJNVVVV/+lymXK6vLedasyz7YSxmueva+yU165jzGWactYxFtcqTE9en2b1f1KfIDAIMlc9v1L5JRVWl9FobWPC9f7EoWd8nujoaC1ZskS33nqrvvrqKy1fvlzLly/3bO/Xr5/uuusu3XnnnQoJCZEkbd68Wf/973/VsWNHvfHGG4qJifE65/33369PP/1UH330kT7++GNdeeWVja7L/dhCXZYtW8ZSigAAAGgQl8vUMUeVyiqrdKyy6vh7Z433P47XvHk+2c2ts6r6z6oTbrZdJ95In+zG+oQxl3nCTfsJYzVvvGuOBdNiYEmxEfXvFCAIDIJMfkmF9heXW12GpXr27Kk1a9bo66+/1ocffqivvvpK//vf/5Sfn6/169fr7rvv1vz58/Xhhx8qLCxMH330kUzT1OWXX14rLHC75JJL9NFHH2nVqlWnFRic6rGFpKSkRp8PAAAA/snlMlXuPPGG3v3eqbLK6p/Lj4//+N5Z5zFllU6VO1ye7RVOl9UfEfVwuoIn/SAwCDJtY8KtLuG0NEXdAwcO1MCBAyVJpmlq3bp1+tvf/qY333xTn332mf7+97/roYce0s6dOyVJ//rXv/Svf/3rlOfMz88/rVqa8rEFAAAANJxpmip3uE76zXzZ8Z+PVTbg5t5x4jHVf7ZEITZDIYZR/afNkM2Q7CE22QxDITbJbrPJ5v7TOL6/zaYQm6r/9IwZP247YcxmGLLbDNls1X/WNWY7XkddY97nr3/Mc506xmzHx+x1jYWcXhN0f0RgEGR8Ma0/GBmGoQEDBui///2vysrK9N5772nhwoV66KGH5HJVp7T9+vVTRkbGKc9zwQUXNEe5AAAALZZpmqpwuqpvxI/fuB+rPP4Nu6NK5cdv0L22OY7f+HvGj4cBDpfnxt9903/MURUw099thhQZZlersBC1Cg1RZFiIWoUd/zM0RK3C7IoMrR5rFRbi9b5VaIjsIbZ6b27dr7rGQgxDISEn3FjXMWYzdNorhcG/ERigxRk1apTee+89FRQUSJI6deokSRoyZIheeOEFK0sDAABodlUuU44qlxxVLjmrTDlc1X/WfO+ocsnpMuWscsnh+bn6vbPKrPHeJcfx/dzHO5zVz9yfatr9j++dOuaoUqDM6DYMnXDDXvvm/sf39ho3+iEnvK/eFnF8X/exYSE2bsRhKQIDBBXTNOv9S3XPnj2SpJSUFEnS5Zdfrj/84Q967733NGPGDEVENKxJSVhYmCTJ6XSeQcUNExoaKofDIafTKbud/9sCAGA183jDNu8b7OqbZYfT+2b6xO3um+7qfeq+6XYev4k/8aa9ru2e4044vtLpfZPvrHXzX33uQPm2/XTV/c189U261837CTf3Xsd57ffjt/rhdm7oEdy480BQeemll/T111/rrrvu0oUXXui1zTRNLViwwDOLYOzYsZKk/v3764YbbtD8+fN1/fXX66WXXlKXLl28jj169KgWLlyokSNHqn379pKktm3bKiwsTPv379fhw4fVpk2bJvtcKSkp+uGHH7Rp06Z6H5sAAKAlMc3qb69Lyp3HXw6VlDtVWvHj++Jyp0rLnSqtcKjSWfOmvu6b7hNvqiurXF7fmFfvH+R32c0oItRW++bd6729jht69/u6v5l3b4sI5YYeOBMEBggqDodDr776ql599VW1bdtW/fv3V2JiogoLC/Xdd99p165dkqSbb75Zt99+u+e4WbNmqbCwUB9//LF69uypjIwMde3aVaZpateuXcrKylJlZaU2b97sCQxCQ0N19dVX6+2331a/fv00dOhQRUZGSpJmzpzp0891ww03aMaMGRo5cqR+8pOfeFZzmD59uhISEnx6LQAAmouzynX8xv7Hm/0ff3aopMb70vIa+1V4BwNV3LzX4m44F2ozqv8MMRQaYpM9xFCorfpPu6163P2cu3u73WZTmL36z5r7hx7fz30+z3bPub2vV3N7RI2b+5phQIQ9RDYbN/SAvyIwQFC5/fbb1bVrVy1ZskRfffWVvvvuOx04cEB2u10dO3bUz372M40fP16XXXaZ13ExMTFavHix5s6dq9dff13ffvut1q9fr9jYWHXo0EE///nPdfXVV6t79+5ex/2///f/lJCQoI8//lhvv/22HA6HJN8HBk888YRsNpveeecdLVy4UJWVlZKkRx55hMAAANDs3E3pit037eX13ORXOOoMBcoq/a+bvP14h3PPTXWNm+ATb569b4pr33RX34y7b8Br3Gx73VTXvsEOO37z7X29uq9VKwQ4/p6bcAC+YJhmsD+15N/69OkjSdq0aVO9+5aXl+uHH35Q165dG/ycPSDxuwMA+FGVy1RphfeU/dJy5483/zXGTzbNv7TCKUeVdf+EDAuxKSbCrpgIu6Ij7IoJD/W8j404/j7crpiImuN2RYeHKtx+8m/LQ0MMpq8DaBEaeh/KDAMAAIAAUeH88Vn90uM38sV1TOUvrXAeH3eqtMbNvzsosFJ0uPtm3n3DX31TH3viTf7x97HuUKDGeERoiKWfAQBaCgIDAACA40zTVJXLlMuUXKZ5/FX93nRJVZ4xU6ap4/tWv3fVONY8flyt7aZ5/BrS0UrvqfzuG/7iE6by13x2v7LKZdl/G7vNOH6TH1rjhj/Uc+N/4rbYiNDjN/o/jkeH2xXCVHkACBgEBgAAtEDOKpe255fqu33FOlrhrHVzW1XjptjlMo//7L4Rrr7hNWvcUFe5TM9NsudG2yWvG26v964ffz7xJv3EG/Eqs+59f6zlxxvyk9bidd0TxmvUEqwiw0JqTdOPOWEqv2f8hKn87pt/lo8DgJaHwAAAgCDnrHJpR/5RbdhbpA25hdqwt0jf5RWr3GHdt9VomBCb4f1tvtdU/pPc5Nd4HxsRqqjwENlDbFZ/FABAACIwAAAgiFS5TO3IL1V2bpE27i3Shr1F2rSvKGjCgRCbIZshGYahEKP6vc0wZBjubdVN62xeP1fvE2L78b3N86chm63G+5ON22pu896ndi2GQmyqt5ZWoSF1TuWvOQOgVWgI3+oDACxDYAAAQICqcpnaeTwc2LC3OiDYtK9Yxxz1L1XXuU2k0lLilJYcp3Yx4bVuZt035bbjN79GjRvqkBo3wjab942290157WNr33yf/Ca9ZgjgrgcAADQfAgMAAAJAlcvUDwW1w4GGrGPfqU0rpSfHq29ynNJT4tS3Y5ziIkOboWoAABDICAwCkGkGcVcmNAl+Z4DA4g4HqnsOFGvD3sJGhQNpyXHV4UByvPomxyo+MqwZqgYAAMGGwCCA2GzVDYuqqur/ByNQk/t3xv07BMB/uFymdhYc9fQb2JBb3XPgaAPCgZTWNcKB4zMHWkcRDgAAAN8gMAggoaGhCgkJ0bFjxxQdHW11OQggJSUlCg0NVWgoU5ABK7lcpn44VB0OuB8t+G5fsUornPUemxxfHQ64+w70TY5TG8IBAADQhAgMAohhGIqMjFRRUZHatGmjkJAQq0tCADh27JiKi4sVHx9PwzCgGblcpnYdOuqZNbDheM+BhoYDfZNjlZ5S3XcgjXAAAABYgMAgwLRr1067du3S7t271aZNG4WHh3MTiFpM01RVVZVKSkpUXFys8PBwJSYmWl0WELRcLlO7D5cdDwcKq8OBvcUqaUA40DEuwmvWQFpynBKiw5uhagAAgFMjMAgwYWFhSklJUUFBgfLy8qwuB34uNDRU8fHxSkxMZEYK4COmaWr3oTJlH1+pYENu9Z8NCQc6xEVUP1Zw/NGCvslxSiQcAAAAforAIABFRkaqc+fOcjqdcjrr/wcqWiabzabQ0FBmoABnwB0OuJcxzM4t0sZ9RSopr//v3qTYH2cOuP8kHAAAAIGEwCCA2e122e38TwgAvmCapva4HyuoMXOguIHhgPtxgvTjMwfaxhAOAACAwMbdJgCgxTFNUzmHj2nD3iJl7y3Uxr1F2ri3WEXHHPUe2z42/PhjBfFKS4lV3+Q4tYuJaIaqAQAAmheBAQAgqJmmqdwjx8OB47MGNuwtalA40C4m3OuRgrTkOLWLJRwAAAAtA4EBACBouMOBjXuLfmxKuLdIhWX1hwNt3eFAjaaE7QkHAABAC0ZgAAAISKZpam/hMW3ILfL0Hdi4t0hHGhAOJEaHe3oNuPsOEA4AAAB4IzAAAPg9dzjgnjHgfrSgYeFAWI1ZA/FKS45T+9hwVhABAACoB4EBAMCvmKapfUXlnlUK3I8WHD5aWe+xCVFhXv0G0lLilBQbQTgAAABwGggMAADNyjRNlVY4dbCkQvk1XgdKyrUlr0Qb9xbpUAPDgb41ljFMS45ThzjCAQAAAF8hMAAA+ISjyqVDpZU6WFLuCQG8QoHSCs+2coerUedu4w4Hko+HAylx6kg4AAAA0KQIDAAAJ2WaporLncovKa81I8ATAhRX/9mQRwYaonVkqGfmgLvvAOEAAABA8yMwAIAWqNLpUn5pRY2ZAOVeQcDBGoFApbNxswFOJa5VqNrFhKut+xUdrnax7vcRSk2IVErrVoQDAAAAfoDAAACChGmaKixznDQIqBkCFDZgdYGGCgux/RgAxITXEQhEqG1MuBKjwxRuD/HZdQEAANC0CAwAwM+VO6o8N/q1+gKUVCjfHQqUVshRZfrsum2iwtQ2uo4QoGYwEB2h2FZ2ZgQAAAAEIQIDALCAy2XqSFnljyFAccVJZwYUlzt9dt1wu636EYDocLWLiTjpzICEqHCF2W0+uy4AAAACD4EBAPjQsUr3bIDyWrMBar4vKK2Q0+Wb2QCGUb3EYKJ7+n903SFAu5hwRYczGwAAAAANQ2AAAPWocpk6fLSyxqoA5bUeDyg4/mdJhe9mA7QKDVG72Bo3/ScJBNpEhckewmwAAAAA+BaBAQDUUFhWqX9+vlPf5RV7ZgMcOlqpKh/NBrAZUkL0iSFA+PEQIMJrVkBUOH9FAwAAwDr8axQAjvt8a74eejtLB4orGn1sdLhd7WLClXjiowCepoHVYUCbqDCF2HgkAAAAAP6PwABAi3esskpPfrxZr67e7TVutxlKPEk/gB9nBkQoMSZMkWH8dQoAAIDgwr9wAbRoWTmFemDueu0sOOoZu6hHov58bV91ah0pG7MBAAAA0EIRGABokRxVLr2wdLteWLbd058gItSmKVf01i2DUllJAAAAAC0egQGAFmdHfqkenLteWblFnrGMlDg9M6afureNtrAyAAAAwH8QGABoMVwuU6+t2a0nP96scodLkhRiMzTxJ2fp18PPUihLEwIAAAAeBAYAWoT9ReV66O0sfbGtwDPWrW2Unr2pnzI6xVtXGAAAAOCnCAwABL33svbpjws3quiYwzN26+BUPXx5b7UKC7GwMgAAAMB/ERgACFqFZZX647ub9H7WPs9Y+9hw/W10hi4+u62FlQEAAAD+j8AAQFD6Ylu+fvtWlg4UV3jGrsroqCeu6aP4yDALKwMAAAACA4EBgKByrLJKT328WXNW7/aMxUbY9efr0nR1RkcLKwMAAAACC4EBgKCRlVOoB+at1878o56xi3ok6q+j09UhrpWFlQEAAACBh8AAQMBzVLn04rLten7pdlW5TElSRKhNky/vrVsGpcpmMyyuEAAAAAg8BAYAAtqO/FI9OHe9snKLPGMZKXF6Zkw/dW8bbWFlAAAAQGCzWV3AmVi7dq2uuOIKxcfHKyoqSoMGDdK8efMadY59+/bpvvvu0znnnKOoqCi1b99eQ4cO1Wuvvaaqqqpa+xuGcdLXhAkTfPTJANTHNE29unqXfvqPLzxhQYjN0H0jeujtuy8kLAAAAADOUMDOMFi2bJkyMzMVERGhsWPHKiYmRvPnz9eYMWOUk5OjSZMm1XuOnTt36oILLtChQ4eUmZmpq666SsXFxVq4cKHGjx+vpUuXatasWbWOS01NrTMc6Nevnw8+GYD67C8q10NvZ+mLbQWesW6JUXpmTD/16xRvXWEAAABAEDFM0zStLqKxnE6nevXqpdzcXK1Zs8Zzo15UVKSBAwdq165d2rp1q1JTU095nnvuuUcvv/yynnvuOd13332e8cLCQmVkZGjPnj3atWuX13kMw9CwYcO0fPlyn3yWPn36SJI2bdrkk/MBwe79rH16ZOFGFR1zeMZuHZyqhy/vrVZhIRZWBgAAAASGht6HBuQjCUuXLtWOHTs0btw4r2/14+LiNGXKFFVWVmrOnDn1nmfnzp2SpCuuuMJrPD4+XkOHDpUkFRQU1DoOQPMrKnPoN/9dp4n/XecJC9rHhuvVXwzU49f0JSwAAAAAfCwgH0lwf7s/atSoWtsyMzMlSStWrKj3PH379tWiRYv00Ucf1Zph8OWXXyopKUnnnHNOreMKCwv1z3/+UwUFBWrTpo2GDBmitLS00/w0AOrzxbZ8PfRWtvYXl3vGrsroqCeu6aP4yDALKwMAAACCV0AGBtu2bZMk9ejRo9a2pKQkRUdHe/Y5lYceekjvv/++HnjgAX3yySdKT0/39DCIjIzUggUL1KpV7bXbs7KydNddd3mNXXbZZZozZ47atWt3mp8KwImOVVZp+idbNHvVLs9YbIRdT1zbV9f0S7auMAAAAKAFCMjAoKiouiN6XFxcndtjY2M9+5xK+/bttXr1at188836+OOP9cknn0iSWrVqpV/96lfKyMiodcykSZN0ww036Oyzz1ZYWJg2btyoJ554Qh9//LGuvPJKrV69WiEhJ58aXVFRoYqKCs/PLpdLhsEa8cCJsnIK9cC89dqZf9QzdlGPRP11dLo6xNUO8gAAAAD4VkD2MPCV7du3a8iQIcrPz9cXX3yhkpIS5eTk6NFHH9UTTzyhESNG1FpaccaMGRo8eLASEhIUExOjwYMH64MPPtCwYcO0du1avfvuu6e85pNPPqm4uDjPa8uWLfRJAGpwVrn098+26fqXV3nCgnC7TY9ddY7m3DaQsAAAAABoJgEZGLhnFpxsFkFxcfFJZx/UNGHCBO3evVvvv/++hg4dqujoaKWkpOjhhx/WxIkTtXr1ar355pv1nsdms+nOO++UJH355Zen3Hfy5MkqKiryvHr16qXExMR6rwG0BDvzS3XDK6v17GdbVeWqXsAlPSVOH/7mIk0Y0lU2G7NxAAAAgOYSkIGBu3dBXX0K9u/fr9LS0jr7G9RUUlKiL7/8Ur1791ZSUlKt7cOHD5ckrVu3rkE1uW/6jx49esr9wsPDFRsb63nZbDYeSUCLZ5qmXl29S1f84wtl5RRKkkJshn4zoofm332hzmoXbW2BAAAAQAsUkIHBsGHDJEmLFy+utW3RokVe+5xMZWWlpJMvm5ifny+p+ga/Ib766itJUpcuXRq0P4BqB4rLdeustXr03U0qd7gkSd0SozT/7gv14KVnKzQkIP+aAgAAAAJeQP5LfMSIEerWrZveeOMNrV+/3jNeVFSkadOmKSwsTOPHj/eM5+XlacuWLV6PMCQkJKhnz57as2ePZs6c6XX+wsJCzZgxQ9KPMw0kacOGDXI4HLXqWbVqlaZPn67Q0FDdeOONvvqYQNB7P2ufRj37uT7fmu8ZGz84VR/+5iL16xRvXWEAAAAAAnOVBLvdrpkzZyozM1MXX3yxxo4dq5iYGM2fP1+7d+/WjBkzvL7pnzx5subMmaNZs2ZpwoQJnvFnn31WV199te688069+eab6t+/v44cOaL33ntP+fn5uuGGGzRy5EjP/k8//bQ+/PBDDR06VJ06dVJoaKg2bdqkxYsXyzAMvfjii+revXsz/pcAAlNRmUN/fHej3sva5xlrFxOuv92YoWFnt7WwMgAAAABuARkYSNXf/K9cuVJTp07V3Llz5XA4lJaWpunTp2vMmDENOsfll1+uVatW6W9/+5tWrlypFStWKCIiQr1799ajjz6qu+++22v/a665RoWFhcrKytKnn36qyspKJSUlaezYsbr//vs1cODApvioQFBZua1Av30rS/uLyz1jV6Z30J+v7av4yDALKwMAAABQk2Gapml1ES1Znz59JEmbNm2yuBKgaR2rrNL0T7Zo9qpdnrHYCLueuLavrumXbF1hAAAAQAvT0PvQgJ1hACBwZOcW6oG567Uj/8dVRIaelai/3ZiuDnGtLKwMAAAAwMkQGABoMs4ql15ctkPPL90mp6t6MlO43abJl/fS+MFdZLOxpCgAAADgrwgMADSJnfmlenBeltbnFHrG0pLj9OyYfjqrXbR1hQEAAABoEAIDAD5lmqZeX7Nbf/los8odLklSiM3Qr4efpYk/OUuhIQG5misAAADQ4hAYAPCZA8XleujtbH2+Nd8z1jUxSs/clKH+nVtbWBkAAACAxiIwAOATH2Tv0yMLN6qwzOEZu2VQqiZf0UuRYfxVAwAAAAQa/hUP4IwUHXNo6rsbtXD9Ps9Yu5hw/XV0ui7p2c7CygAAAACcCQIDAKfty+0F+u1bWcorKveM/TS9g/58TV+1jgqzsDIAAAAAZ4rAAECjlTuq9NTHWzR71S7PWEyEXX++tq+uzugow2C5RAAAACDQERgAaJQNuUW6f+467cg/6hkbclaC/jY6Qx3jW1lYGQAAAABfIjAA0CDOKpdeWr5D/1iyTU6XKUkKt9v08OW9dOvgLrLZmFUAAAAABBMCAwD12plfqgfnZWl9TqFnLC05Ts+OydBZ7WKsKwwAAABAkyEwAHBSpmnq9a/2aNqHm3XMUSVJCrEZ+vUl3TVxRA+FhtgsrhAAAABAUyEwAFCnA8Xl+t3b2VqxNd8z1jUxSs/clKH+nVtbWBkAAACA5kBgAKCWD7Pz9IeFG1RY5vCM3Tyos6Zc0VuRYfy1AQAAALQE/MsfgEfRMYemvrtRC9fv84y1iwnXX0en65Ke7SysDAAAAEBzIzAAIEn6cnuBfvtWlvKKyj1jP03roD9f21eto8IsrAwAAACAFQgMgBau3FGl6Z9s0awvd3nGYiLseuKavrqmX0cZBsslAgAAAC0RgQHQgm3ILdID89Zr+8FSz9iF3RM048YMdYxvZWFlAAAAAKxGYAC0QM4ql15evkN/X7JNTpcpSQq32/T7y3ppwoVdZLMxqwAAAABo6QgMgBbmh4KjenDeeq3bU+gZ65scq+fG9NNZ7WKsKwwAAACAXyEwAFoI0zT1n6/26C8fbtYxR5UkyWZIvx5+lib+pIfC7DaLKwQAAADgTwgMgBbgYHG5fjc/W8u/z/eMdUmI1DNj+mlA59YWVgYAAADAXxEYAEHuw+w8/WHhBhWWOTxjNw/qrClX9FZkGH8FAAAAAKgbdwtAkCo65tBj723SgnV7PWNtY8L119HpGt6znYWVAQAAAAgEBAZAEFq1vUC/fStL+4rKPWM/TeugP1/bV62jwiysDAAAAECgIDAAgki5o0p//eR7/fvLHzxjMRF2PXFNX13Tr6MMg+USAQAAADQMgQEQJDbuLdL9c9dr+8FSz9iF3RM048YMdYxvZWFlAAAAAAIRgQEQ4JxVLr2yYoee+2ybnC5TkhRut+n3l/XShAu7yGZjVgEAAACAxiMwAALYroKjemDeeq3bU+gZ65scq2dv6qce7WOsKwwAAABAwCMwAAKQaZp64+s9+vMHm3XMUSVJshnSr4efpYk/6aEwu83iCgEAAAAEOgIDIMAcLC7X7+dna9n3+Z6xLgmRevqmfjo3tbWFlQEAAAAIJgQGQAD5aEOe/rBgg46UOTxjP7+gs/7w096KDOP/zgAAAAB8hzsMIAAUlzv02Lub9M66vZ6xtjHh+usN6Rreq52FlQEAAAAIVgQGgJ9btb1Av30rS/uKyj1jV6Ql6c/XpqlNVJiFlQEAAAAIZgQGgJ8yTVNPfbxF/+/znZ6xmHC7/nRtH13bL1mGwXKJAAAAAJoOgQHgpz797oBXWDC4W4Jm3JSh5PhWFlYFAAAAoKUgMAD81MrtBZ7394/sod/8pIdsNmYVAAAAAGgeLNYO+Kms3CLP+9uGdCUsAAAAANCsCAwAP1TpdGnzvmJJUrfEKMW1CrW4IgAAAAAtDYEB4Ie+31+iyiqXJCk9Jc7iagAAAAC0RAQGgB/Kyi30vE9PibesDgAAAAAtF4EB4IeyvQIDZhgAAAAAaH4EBoAfyj7e8DDEZqhPRwIDAAAAAM2PwADwM2WVTm09UCJJ6tEuWq3CQiyuCAAAAEBLRGAA+JlN+4rlMqvfZ9C/AAAAAIBFCAwAP5OVU+h5n96JxxEAAAAAWIPAAPAz7v4FEjMMAAAAAFiHwADwM+4VEsLsNvVMirG2GAAAAAAtFoEB4EeKyhzadahMknROh1iFhvB/UQAAAADW4G4E8CPZews97zNS6F8AAAAAwDoEBoAfqdm/IJ3+BQAAAAAsRGAA+JGaKyRksEICAAAAAAsRGAB+xD3DIDrcrm6J0RZXAwAAAKAlIzAA/MTB4nLtLy6XJPVNjpXNZlhcEQAAAICWjMAA8BNZNfoXZNC/AAAAAIDFCAwAP5GdW+h5T8NDAAAAAFYjMAD8RJbXCgk0PAQAAABgLQIDwA+YpumZYdAmKkwprVtZWxAAAACAFo/AAPADOYePqbDMIal6doFh0PAQAAAAgLUIDAA/kEX/AgAAAAB+hsAA8AM1Gx5m0L8AAAAAgB8gMAD8gHfDw3jrCgEAAACA4wgMAItVuUxt3FsdGHSMi1DbmHCLKwIAAACAAA8M1q5dqyuuuELx8fGKiorSoEGDNG/evEadY9++fbrvvvt0zjnnKCoqSu3bt9fQoUP12muvqaqqqs5jFi1apGHDhikmJkaxsbEaPny4lixZ4ouPhBZoR36pyiqrf9eYXQAAAADAX9itLuB0LVu2TJmZmYqIiNDYsWMVExOj+fPna8yYMcrJydGkSZPqPcfOnTt1wQUX6NChQ8rMzNRVV12l4uJiLVy4UOPHj9fSpUs1a9Ysr2Nef/113XLLLWrbtq0mTJggSZo7d64uvfRSzZs3T6NHj26Kj4sglpVT6Hmf3on+BQAAAAD8g2Gapml1EY3ldDrVq1cv5ebmas2aNerXr58kqaioSAMHDtSuXbu0detWpaamnvI899xzj15++WU999xzuu+++zzjhYWFysjI0J49e7Rr1y7PeY4cOaJu3brJbrdr3bp1SklJkSTl5uaqf//+kqpDiJiYmAZ/lj59+kiSNm3a1OBjEFwefXejXl29W5L0nzsu0JCzEi2uCAAAAEAwa+h9aEA+krB06VLt2LFD48aN84QFkhQXF6cpU6aosrJSc+bMqfc8O3fulCRdccUVXuPx8fEaOnSoJKmgoMAz/tZbb6mwsFATJ070hAWSlJKSonvvvVcFBQVasGDBmXw0tEA1Gx72TWaGAQAAAAD/EJCBwfLlyyVJo0aNqrUtMzNTkrRixYp6z9O3b19J0kcffeQ1XlhYqC+//FJJSUk655xzfH5dwK3S6dLmfcWSpG6JUYprFWpxRQAAAABQLSB7GGzbtk2S1KNHj1rbkpKSFB0d7dnnVB566CG9//77euCBB/TJJ58oPT3d08MgMjJSCxYsUKtWrRp0XfdYfdetqKhQRUWF52eXyyXDMOqtFcHp+/0lqqxySZLSU5hdAAAAAMB/BGRgUFRUPYU7Lq7uG6zY2FjPPqfSvn17rV69WjfffLM+/vhjffLJJ5KkVq1a6Ve/+pUyMjIafN3Y2FivfU7mySef1OOPP+411rZt23prRXDKyi30vGeFBAAAAAD+JCAfSfCV7du3a8iQIcrPz9cXX3yhkpIS5eTk6NFHH9UTTzyhESNGnHRpxdM1efJkFRUVeV69evVSYiJN7lqq7BqBQQYrJAAAAADwIwE5w8D9Df/Jvs0vLi5W69at6z3PhAkTtHv3bu3cuVNJSUmSpOjoaD388MM6cOCAnnvuOb355pv6+c9/Xuu6CQkJta5Zc5+TCQ8PV3h4uOdnm61FZzYtXvbxhochNkPndCAwAAAAAOA/AvJu9VT9Avbv36/S0tI6+wzUVFJSoi+//FK9e/f2hAU1DR8+XJK0bt26Bl33VP0NgLqUVTq19UCJJOns9jFqFRZicUUAAAAA8KOADAyGDRsmSVq8eHGtbYsWLfLa52QqKysleS+bWFN+fr4kec0G8MV1AbdN+4rlMqvfZ9DwEAAAAICfCcjAYMSIEerWrZveeOMNrV+/3jNeVFSkadOmKSwsTOPHj/eM5+XlacuWLV6PMCQkJKhnz57as2ePZs6c6XX+wsJCzZgxQ9KPMw0k6aabblJcXJyef/555ebmesZzc3P1wgsvKDExUdddd52vPy6CVFZOoec9DQ8BAAAA+JuADAzsdrtmzpwpl8uliy++WL/85S81adIkZWRkaOvWrZo2bZq6dOni2X/y5Mnq3bu3FixY4HWeZ599Vna7XXfeeadGjhyphx56SHfccYfOPvtsbdmyRTfccINGjhzp2b9169Z64YUXVFBQoAEDBmjixImaOHGiBgwYoEOHDumll15STExMc/1nQIBz9y+QWFIRAAAAgP8JyKaHUvU3/ytXrtTUqVM1d+5cORwOpaWlafr06RozZkyDznH55Zdr1apV+tvf/qaVK1dqxYoVioiIUO/evfXoo4/q7rvvrnXMzTffrMTERE2bNk2zZs2SYRg699xz9cgjj3iFC0B93CskhNtt6plE0AQAAADAvximaZpWF9GS9enTR5K0adMmiytBcyoqcyjjT9W9MPp3jteCe4ZYXBEAAACAlqKh96EB+UgCEOiy9xZ63mfQvwAAAACAHyIwACxA/wIAAAAA/o7AALAAKyQAAAAA8HcEBoAF3DMMYsLt6pYYZXE1AAAAAFAbgQHQzA4Wl2t/cbkkqW9ynGw2w+KKAAAAAKA2AgOgmWXV7F/Qif4FAAAAAPwTgQHQzLJzCz3vWSEBAAAAgL8iMACaWRYrJAAAAAAIAAQGQDMyTdMzwyAhKkzJ8a2sLQgAAAAAToLAAGhGOYePqbDMIal6doFh0PAQAAAAgH8iMACaUVaN/gXp9C8AAAAA4McIDIBm5NXwkBUSAAAAAPgxAgOgGdVseJiWHG9dIQAAAABQDwIDoJlUuUxt3FsdGHSMi1DbmHCLKwIAAACAkyMwAJrJjvxSlVVWSaJ/AQAAAAD/R2AANJOsnELP+3T6FwAAAADwcwQGQDPJrtG/IIMZBgAAAAD8HIEB0ExqrpDQN5kZBgAAAAD8G4EB0AwqnS5tziuRJHVLjFJcq1CLKwIAAACAUyMwAJrBlv3FqqxySZLSU5hdAAAAAMD/ERgAzSCrRv8CVkgAAAAAEAgIDIBmkF1jhYQMVkgAAAAAEAAIDIBm4F4hIcRm6JwOBAYAAAAA/B+BAdDEyiqd2nawuuHh2e1j1CosxOKKAAAAAKB+BAZAE9u4t1gus/p9Bg0PAQAAAAQIAgOgiWXnFnre0/AQAAAAQKAgMACamPcKCcwwAAAAABAYCAyAJuaeYRBut6lnUoy1xQAAAABAAxEYAE2osKxSuw+VSZLO6Rir0BD+LwcAAAAgMHD3AjSh7BqPI2TQvwAAAABAACEwAJqQd8ND+hcAAAAACBwEBkAT8m54GG9dIQAAAADQSAQGQBPacDwwiAm3q1tilMXVAAAAAEDDERgATeRgcbn2F5dLkvomx8lmMyyuCAAAAAAajsAAaCJejyN0on8BAAAAgMBCYAA0kZoND1khAQAAAECgITAAmoh3w0NmGAAAAAAILAQGQBMwTdMzwyAhKkzJ8a2sLQgAAAAAGonAAGgCOYePqbDMIal6doFh0PAQAAAAQGAhMACaQFaN/gXp9C8AAAAAEIAIDIAm4NXwkBUSAAAAAAQgAgOgCXg3PIy3rhAAAAAAOE0EBoCPVblMbdxbHRgkx7dSYnS4xRUBAAAAQOMRGAA+tiO/VGWVVZJYThEAAABA4CIwAHwsK6fQ857HEQAAAAAEKgIDwMeya/QvyGCGAQAAAIAARWAA+FjNFRL6EhgAAAAACFAEBoAPVTpd2pxXIknq1jZKsRGhFlcEAAAAAKeHwADwoS37i1VZ5ZIkZdC/AAAAAEAAIzAAfCirRv8CVkgAAAAAEMgIDAAfymaFBAAAAABBgsAA8CH3Cgl2m6E+HWMtrgYAAAAATh+BAeAjZZVObTtY3fDw7PYxiggNsbgiAAAAADh9BAaAj2zcWyyXWf0+oxP9CwAAAAAENgIDwEeycws97+lfAAAAACDQERgAPsIKCQAAAACCCYEB4CPuGQbhdpvObh9jbTEAAAAAcIYIDAAfKCyr1O5DZZKkPh1jFRrC/7UAAAAABDbuagAfyPZ6HCHeukIAAAAAwEcIDAAfqNnwkBUSAAAAAAQDAgPAB7KYYQAAAAAgyBAYAD7gnmEQE25X14Qoa4sBAAAAAB8gMADO0IHich0orpAk9U2Ok81mWFwRAAAAAJw5AgPgDGXlFHrep9O/AAAAAECQCOjAYO3atbriiisUHx+vqKgoDRo0SPPmzWvw8V26dJFhGKd8ffHFF17HnGrfCRMm+PgTIhDUXCEhg/4FAAAAAIKE3eoCTteyZcuUmZmpiIgIjR07VjExMZo/f77GjBmjnJwcTZo0qd5z3H///SosLKw1XlBQoBdffFGtW7fW+eefX2t7ampqneFAv379TuOTINBl1VghIT2FGQYAAAAAgkNABgZOp1N33nmnbDabPv/8c8+N+qOPPqqBAwdqypQpGj16tFJTU095nvvvv7/O8aefflqSdPPNNysiIqLW9i5duuixxx47k4+AIGGapjbsrZ5hkBAVpuT4VhZXBAAAAAC+EZCPJCxdulQ7duzQuHHjvL7Vj4uL05QpU1RZWak5c+ac9vn/9a9/SZJuv/32My0VQW7P4TIVljkkVc8uMAwaHgIAAAAIDgE5w2D58uWSpFGjRtXalpmZKUlasWLFaZ171apV2rx5s8477zxlZGTUuU9hYaH++c9/qqCgQG3atNGQIUOUlpZ2WtdDYMuq0b8gnf4FAAAAAIJIQAYG27ZtkyT16NGj1rakpCRFR0d79mks9+yCO+6446T7ZGVl6a677vIau+yyyzRnzhy1a9fulOevqKhQRUWF52eXy8W30gEsu8YKCRmskAAAAAAgiATkIwlFRdXf6sbF1X2DFhsb69mnMUpLSzVv3jxFRkbqZz/7WZ37TJo0SatWrVJBQYGKi4u1atUqXX755frkk0905ZVXqqqq6pTXePLJJxUXF+d5bdmyRQUFBY2uFf4hmxkGAAAAAIJUQAYGTWXu3LkqLS3VjTfeqNjY2Dr3mTFjhgYPHqyEhATFxMRo8ODB+uCDDzRs2DCtXbtW77777imvMXnyZBUVFXlevXr1UmJiYlN8HDSxKpepjfuqA4Pk+FZKjA63uCIAAAAA8J2ADAzcMwtONouguLj4pLMPTqUhjyPUxWaz6c4775Qkffnll6fcNzw8XLGxsZ6XzWbjkYQAtf1gqcoqq2eUsJwiAAAAgGDjk8Dg7bfflsPh8MWpGsTdu6CuPgX79+9XaWlpnf0NTuW7777T6tWr1atXLw0dOrTRNblnCRw9erTRxyIwZeUWet7zOAIAAACAYOOTwOCmm25Sx44d9cADDyg7O9sXpzylYcOGSZIWL15ca9uiRYu89mmoM11K8auvvpIkdenS5bSOR+DJrhEYZDDDAAAAAECQMUzTNM/0JDZbde7gnlrfv39/3X777Ro3btxpPRpQH6fTqZ49e2rv3r1as2aN+vXrJ6n6EYWBAwdq165d+v777z0373l5eSoqKlKHDh3qrMfhcCg5OVmFhYXKzc096UoHGzZsUK9evRQaGuo1vmrVKl166aVyOBzavHmzunfv3uDP0qdPH0nSpk2bGnwM/MPVL6z0ND3MfmyUYiNC6zkCAAAAAKzX0PtQn8wweO+993TdddfJbrfLNE3973//07333qsOHTro5z//uT777DNfXMbDbrdr5syZcrlcuvjii/XLX/5SkyZNUkZGhrZu3app06Z5fdM/efJk9e7dWwsWLDhp/fn5+brqqqtOuSzi008/rY4dO+q6667Tb37zG02aNEmXXXaZhg4dqvLycv3jH/9oVFiAwFXhrNLmvGJJUre2UYQFAAAAAIKO3RcnufLKK3XllVeqoKBAr732mmbPnq0NGzaovLxcb775pt5880117txZEyZM0IQJE5SamnrG1xw+fLhWrlypqVOnau7cuXI4HEpLS9P06dM1ZsyYRp2roc0Or7nmGhUWFiorK0uffvqpKisrlZSUpLFjx+r+++/XwIEDT/vzILB8v79EjqrqyTkZ9C8AAAAAEIR88khCXf73v//p3//+t/773//qyJEj1RczDBmGoUsuuUS33367rr/+eoWHt+yl6HgkITC9tma3/rhwoyRp6lXn6LYhXS2uCAAAAAAaplkfSajLgAED9MILLygvL09vvvmmMjMzZRiGXC6Xli5dqptvvlkdOnTQr3/9a33zzTdNVQbQJLJzCj3vWSEBAAAAQDBqssDALSwsTDfddJM+/vhj7d69W3/+85/Vvn17maapwsJCvfLKK7rggguUkZGhV155RRUVFU1dEnDG3M0O7TZDfTrGWlwNAAAAAPhekwcGbmVlZfrss8+0ePFiHTx40LOigmmaMk1TGzZs0K9//Wt169btpM0JAX9QVunUtoMlkqSz28coIjTE4ooAAAAAwPeaPDBYuXKlbr/9diUlJekXv/iFvvjiC5mmqdjYWN19991atWqV/vnPf2rQoEEyTVN5eXkaPXq0Pvnkk6YuDTgtG/cWy3W880dGJ98vGwoAAAAA/sAnqyScaO/evZozZ45mz56tHTt2SKqeSSBJF110ke644w7deOONioiIkCQNGjRId9xxhz7//HPdeuut2r17t/7yl7/osssua4rygDOSnVvoeU//AgAAAADBymeBQWVlpRYsWKBZs2ZpyZIlcrlcnpCgffv2Gj9+vO644w716NHjpOe4+OKL9cwzz+iGG27Qhg0bfFUa4FNZx/sXSFJ6CjMMAAAAAAQnnwQG99xzj+bOnavCwkJJ1bMJbDabLrvsMt1xxx266qqrZLc37FJ9+/aVJJWUlPiiNMDn3DMMwu02nd0+xtpiAAAAAKCJ+CQweOWVVzzvU1NT9Ytf/EK/+MUvlJyc3OhzhYeHq3PnzrLZmq0fI9BghWWV2n2oTJLUp2OsQkP4PQUAAAAQnHwSGISGhuraa6/VHXfcoZEjR3pWQDgdnTt31q5du3xRFuBz2V6PI8RbVwgAAAAANDGfBAb79u1TQkKCL04F+LWaDQ9ZIQEAAABAMPPJfGrCArQUWcwwAAAAANBC+GyVhD179kiqXhEhPDz8lPuWl5fr4MGDkqofQQAChXuGQUy4XV0ToqwtBgAAAACakE9mGCxevFhdu3ZVWlqaysrK6t2/rKxMffr0Ubdu3bR8+XJflAA0uQPF5TpQXCFJSkuJk812+r06AAAAAMDf+SQweOutt2Sapq699lq1bt263v3btGmjG264QS6XS3PnzvVFCUCTy8op9LzncQQAAAAAwc4ngcHq1atlGIZGjRrV4GMyMzM9xwKBoOYKCRkpNDwEAAAAENx8Ehi4l0E8++yzG3zMWWedJUn64YcffFEC0OSyaqyQkN4p3rI6AAAAAKA5+CQwcDqdkqSQkJAGH+Pet7y83BclAE3KNE1t2Fs9wyAxOkwd4yIsrggAAAAAmpZPAoPExERJ0s6dOxt8jHvfNm3a+KIEoEntOVymwjKHpOr+BYZBw0MAAAAAwc0ngUG/fv0kqVENDN98801JUt++fX1RAtCksmr0L0infwEAAACAFsAngcE111wj0zT1zjvv6K233qp3/3nz5umdd96RYRi69tprfVEC0KSya6yQkMEKCQAAAABaAJ8EBrfeequ6dOki0zQ1btw4/fa3v1VOTk6t/XJycvTggw/q5z//uQzDUKdOnXTHHXf4ogSgSWUzwwAAAABAC2P3xUnCwsL0zjvv6OKLL1ZpaameffZZPfvss+rcubM6dOggScrLy9OePXskVTeQi46O1oIFCxQeHu6LEoAmU+UytXFfdWCQHN9KCdH8zgIAAAAIfj6ZYSBV9zH46quv1L9/f5mmKdM0tXv3bn311Vf66quvtHv3bs/4ueeeq6+//lr9+/f31eWBJrP9YKnKKqskSRmdmF0AAAAAoGXwyQwDt969e+vbb7/Vp59+qg8++EDr1q1TQUGBpOqVFAYMGKCrrrpKI0aM8OVlgSaVlVvoeZ9O/wIAAAAALYRPAwO3Sy+9VJdeemlTnBpodtlegQEzDAAAAAC0DD57JAEIVu6Gh4YhpSUTGAAAAABoGQgMgFOocFZpc16xJKlbYpRiIkItrggAAAAAmkeTPJIgSVVVVTpy5IiOHTsm0zRPuW/nzp2bqgzgjGzJK5Gjqvr3N4P+BQAAAABaEJ8GBgUFBXr++ee1cOFCfffdd3K5XPUeYxiGnE6nL8sAfIb+BQAAAABaKp8FBqtWrdL111+v/Pz8emcUAIEi63j/AklK7xRvXSEAAAAA0Mx8EhgcOnRI11xzjQ4dOqTo6Gjdcccdio+P12OPPSbDMDRz5kwdPnxY33zzjd577z2Vl5dryJAhuv32231xeaDJuGcY2G2GzukQa20xAAAAANCMfBIYvPDCCzp06JDCw8O1evVq9enTR5s2bdJjjz0mSbrttts8++bl5WncuHH6/PPPNXjwYE2fPt0XJQA+d7TCqe0HSyVJZ7ePUURoiMUVAQAAAEDz8ckqCR9//LEMw9AvfvEL9enT55T7dujQQR999JG6d++uGTNmaOnSpb4oAfC5jXuL5Dr+dE1GJ/oXAAAAAGhZfBIYbN++XZI0cuRIz5hhGJ73VVVVXvu3atVKDzzwgEzT1CuvvOKLEgCfy67Zv4AVEgAAAAC0MD4JDIqLq9epT01N9YxFRER43peUlNQ65rzzzpMkffXVV74oAfC5LFZIAAAAANCC+SQwiI6OliSv5RHbtGnjeb9r165ax5SXl0uSDh486IsSAJ9zzzAIt9t0dvsYi6sBAAAAgOblk8DgrLPOkiTt2bPHMxYfH6+kpCRJ0rJly2ods3LlSklSVFSUL0oAfOrI0UrtOVwmSerTMVahIT75vwoAAAAABAyf3AVdcMEFkqS1a9d6jV922WUyTVN//etftW3bNs/4mjVr9Le//U2GYej888/3RQmAT2XvpX8BAAAAgJbNJ4FBZmamTNPUO++84zX+4IMPym636+DBg+rTp4/OP/98nXPOObroootUWFgoSbrvvvt8UQLgU9k5hZ73rJAAAAAAoCXyWWAwfvx4DRo0SD/88INnvG/fvnr55ZcVEhIip9Opb7/9Vlu2bPGsmvDYY4/psssu80UJgE9lsUICAAAAgBbO7ouThIaGavbs2XVuu/322zV06FDNnj1bmzZtktPpVI8ePXTLLbd4VkoA/E328RUSYsLt6ppAnw0AAAAALY9PAoP69OzZU08++WRzXAo4Y/uLynWwpEKSlJYSJ5vNsLgiAAAAAGh+PgkM/vSnP0mqbn6YmZnpi1MClsk6PrtA4nEEAAAAAC2XTwKDxx57TIZhaMGCBb44HWCpDTX6F2Sk0PAQAAAAQMvkk6aHCQkJkqTOnTv74nSApbxmGHSKt6wOAAAAALCSTwKDs846S5K0f/9+X5wOsIxpmtqwt3qGQWJ0mDrGRVhcEQAAAABYwyeBwZgxY2SapubNm+eL0wGW2XO4TIVlDknV/QsMg4aHAAAAAFomnwQG99xzjzIyMvTqq6+edHlFIBBk1ehfkE7/AgAAAAAtmE+aHu7fv18zZ87U7bffrttvv11vvPGGxo0bp/T0dLVu3VohISGnPJ7eB/AX2TmFnvcZrJAAAAAAoAXzSWDQpUsXz9Rt0zS1ZMkSLVmypEHHGoYhp9PpizKAM5bNDAMAAAAAkOSjwECqDgrqeg8EiiqXqY37qgOD5PhWSogOt7giAAAAALCOTwKDWbNm+eI0gKW2HyxVWWWVJCmjE7MLAAAAALRsPgkMbr31Vl+cBrBUVm6h5306/QsAAAAAtHA+WSUBCAbZXoEBMwwAAAAAtGwEBsBx7oaHhiGlJRMYAAAAAGjZCAwASRXOKm3OK5YkdUuMUkxEqMUVAQAAAIC1fNLD4Be/+MVpH2sYhv71r3/5ogzgtG3JK5Gjqnp1jwz6FwAAAACAbwKD2bNnyzCMRh9nmiaBAfwC/QsAAAAAwJtPAoPOnTvXGxgcPXpUhw4d8oQEiYmJioyM9MXlgTOWdbx/gSSld4q3rhAAAAAA8BM+CQx27drVoP2OHDmi//73v3r00UcVHx+v9957Tz179vRFCcAZcc8wsNsMndMh1tpiAAAAAMAPNGvTw9atW+uee+7Rl19+qYMHD+ryyy/XkSNHmrMEoJajFU5tP1gqSeqZFKOI0BCLKwIAAAAA61mySkLPnj31m9/8Rrt27dLTTz9tRQmAx8a9RXJV9ztUOg0PAQAAAECShcsqjhw5UpL0zjvvWFUCIEnKrtG/IIOGhwAAAAAgycLAIDo6WpK0Z88eq0oAJElZXiskxFtWBwAAAAD4E8sCg3Xr1kmSQkNDT/sca9eu1RVXXKH4+HhFRUVp0KBBmjdvXoOP79KliwzDOOXriy++qHXcokWLNGzYMMXExCg2NlbDhw/XkiVLTvtzwFruGQYRoTad3T7a4moAAAAAwD/4ZJWExvrhhx/02GOPyTAM9evX77TOsWzZMmVmZioiIkJjx45VTEyM5s+frzFjxignJ0eTJk2q9xz333+/CgsLa40XFBToxRdfVOvWrXX++ed7bXv99dd1yy23qG3btpowYYIkae7cubr00ks1b948jR49+rQ+D6xx5Gil9hwukyT16Rgne4hlGRoAAAAA+BXDNE3zTE/y6quv1ruPy+XSkSNH9M033+jdd99VWVmZDMPQG2+8oTFjxjTqek6nU7169VJubq7WrFnjCR2Kioo0cOBA7dq1S1u3blVqaurpfBw9/fTT+u1vf6uJEyfqH//4h2f8yJEj6tatm+x2u9atW6eUlBRJUm5urvr37y9J2rlzp2JiYhp8rT59+kiSNm3adFq14sys2JqvW//9tSTptiFdNPWqPhZXBAAAAABNq6H3oT6ZYTBhwgQZhtHg/d0ZxW9+85tGhwWStHTpUu3YsUO33Xab1wyFuLg4TZkyRRMmTNCcOXP06KOPNvrckvSvf/1LknT77bd7jb/11lsqLCzU448/7gkLJCklJUX33nuvHnvsMS1YsEDjx48/reui+WXnFHreZ9C/AAAAAAA8fDb/2jTNBr3i4uJ09dVX65NPPtGzzz57Wtdavny5JGnUqFG1tmVmZkqSVqxYcVrnXrVqlTZv3qzzzjtPGRkZzXZdWCOrxgoJ6ayQAAAAAAAePplh8MMPP9S7j81mU0xMjOLj48/4etu2bZMk9ejRo9a2pKQkRUdHe/ZpLPfsgjvuuKNR13WP1XfdiooKVVRUeH52uVyNmp0B38o+vkJCTIRdXRKirC0GAAAAAPyITwKD0+0VcLqKiqq/FY6Lq/sb4djYWM8+jVFaWqp58+YpMjJSP/vZzxp13djYWK99TubJJ5/U448/7jXWtm3bRteKM7e/qFwHS6rDm/SUONlsBDcAAAAA4EZL+Brmzp2r0tJS3XjjjZ4AwNcmT56soqIiz6tXr15KTExskmvh1LKOzy6QpHT6FwAAAACAF0uWVTxT7m/4T/ZtfnFxsVq3bt3o857qcYQTr5uQkFDrmjX3OZnw8HCFh4d7frbZyGyskl0jMMigfwEAAAAAePHJ3eoPP/ygn/zkJxoxYoT27t1b7/579+7ViBEjGrz/iU7VL2D//v0qLS2ts8/AqXz33XdavXq1evXqpaFDhzb6uqfqbwD/lO3V8DDeukIAAAAAwA/5JDB49dVXtXz5clVWVio5Obne/ZOTk+V0OrV8+XK99tprjb7esGHDJEmLFy+utW3RokVe+zTUyZZSbOrrwhqmaXoCg8TocHWIi7C4IgAAAADwLz4JDJYsWSLDMHT99dc3+Jjrr79epmnWefNdnxEjRqhbt2564403tH79es94UVGRpk2bprCwMI0fP94znpeXpy1btpz0EQaHw6HXXntNoaGhXsed6KabblJcXJyef/555ebmesZzc3P1wgsvKDExUdddd12jPw+a3+5DZSo65pBU/TgCK1UAAAAAgDefBAabN2+WJA0YMKDBx/Tr109S9aMAjWW32zVz5ky5XC5dfPHF+uUvf6lJkyYpIyNDW7du1bRp09SlSxfP/pMnT1bv3r21YMGCOs/33nvvKT8/X1dddZXatWt30uu2bt1aL7zwggoKCjRgwABNnDhREydO1IABA3To0CG99NJLiomJafTnQfOj4SEAAAAAnJpPmh66v7mPj49v8DHufY8cOXJa1xw+fLhWrlypqVOnau7cuXI4HEpLS9P06dM1ZsyYRp2rvmaHNd18881KTEzUtGnTNGvWLBmGoXPPPVePPPKIRo4ceVqfBc3Pq39BJxoeAgAAAMCJfBIYxMbG6siRIzp06FCDj3HvGxkZedrXHThwoD7++ON695s9e7Zmz5590u0fffRRo6572WWX6bLLLmvUMfAvNVdISE8mMAAAAACAE/nkkQT39P/ly5c3+Jhly5ZJkjp37uyLEoAGc1a5tHFv9TKYyfGtlBAdXs8RAAAAANDy+CQwGDlypEzT1Isvvqi8vLx699+7d69efPFFGYbBNH40u+35pTrmqJIkZfA4AgAAAADUySeBwd13363Q0FAVFhZqxIgRys7OPum+WVlZGjlypAoLC2W323XPPff4ogSgwbJzavQvoOEhAAAAANTJJz0MUlNT9Ze//EW/+93v9P3332vAgAG65JJLdNFFF6lDhw6Sqpc2/Pzzz7VixQqZpinDMPT444+re/fuvigBaDDvFRKYYQAAAAAAdfFJYCBJv/3tb3Xs2DE9/vjjcrlcWrZsmadPQU2macpms+nxxx/Xww8/7KvLAw3mXiHBMKQ0Gh4CAAAAQJ188kiC2x//+Ed98803Gjt2rOLi4mSaptcrLi5OP//5z/Xtt9/qD3/4gy8vDTRIhbNKW/ZXNzzslhilmIhQiysCAAAAAP/ksxkGbv369dMbb7wh0zT1ww8/qKCgQJKUmJiorl27yjAMX18SaLDNeSVyVJmSpAz6FwAAAADASfk8MHAzDEPdunVTt27dmuoSQKNl078AAAAAABrEp48kAP4uq+YKCZ3irSsEAAAAAPycT2YYFBUV6e9//7sk6c477/SsjHAyeXl5+r//+z9J0qRJkxQVFeWLMoB6bdhbKEmy2wyd0yHW2mIAAAAAwI/5JDD4z3/+o8cee0w9evTQo48+Wu/+SUlJ+s9//qPt27crOTlZt99+uy/KAE7paIVT2w+WSpJ6JsUoIjTE4ooAAAAAwH/55JGEjz/+WIZh6KabbmrQ/oZhaOzYsTJNU++//74vSgDqtXFvkVzV/Q6VTsNDAAAAADglnwQG69evlyRdeOGFDT5m8ODBXscCTS0798f+BRk0PAQAAACAU/JJYHDw4EFJqrd3QU1JSUmSpAMHDviiBKBeWV4rJMRbVgcAAAAABAKfBAYRERGSpLKysgYf4943JITnyNE83DMMIkJtOrt9tMXVAAAAAIB/80lg4J5Z8M033zT4GPe+7pkGQFM6crRSew5Xh1R9OsbJHsKKogAAAABwKj65a7roootkmqZeeuklORyOevd3OBx66aWXZBiGhg4d6osSgFPK3vtj/4J0+hcAAAAAQL18EhjcdtttkqRt27Zp3Lhxp3w0oaysTD/72c+0detWr2OBppSdU+h5n0H/AgAAAACol90XJ7nwwgs1duxYvfnmm3rnnXf09ddf684779RFF13keVwhLy9Pn3/+uWbOnKnc3FwZhqHRo0dr2LBhvigBOKWsXGYYAAAAAEBj+CQwkKR///vfKigo0Geffabc3FxNnTq1zv1M05QkXXrppZozZ46vLg+cUvbxFRJiIuzqkhBlbTEAAAAAEAB81vktIiJCixYt0nPPPafk5GSZplnnq1OnTvrHP/6hTz75xLO6AtCU9heV62BJhaTq2QU2m2FxRQAAAADg/3w2w0CSDMPQb37zG02cOFHr16/XunXrVFBQIElKTEzUgAEDlJGRIcPghg3NJ+v47AJJSqd/AQAAAAA0iE8DAzfDMNS/f3/179+/KU4PNEp2jcAgg/4FAAAAANAgLEaPoJft1fAw3rpCAAAAACCA+HyGgWmaWr9+vbKyslRQUKBjx455Gh2ezKOPPurrMgBJ1b+P7sAgMTpcHeLomwEAAAAADeHTwGDOnDl6/PHHtXv37kYdR2CAprL7UJmKjjkkVT+OQP8MAAAAAGgYnwUGf/jDH/TUU0/VO5tAqu5x0JD9gDNFw0MAAAAAOD0+6WHw1Vdf6cknn5QkXXrppVq/fr3+97//SaoOB6qqqpSfn6+PP/5YV199tUzT1NChQ5WXlyeXy+WLEoA6efUv6ETDQwAAAABoKJ8EBi+//LIkKTU1VR9++KHS09MVGhrq2W4YhhISEpSZmamFCxfqxRdf1MqVK3XZZZepsrLSFyUAdfJeISHesjoAAAAAIND4JDBYtWqVDMPQb37zG9nt9T/lcPfdd+uGG25Qdna2XnrpJV+UANTirHJp495iSVJK61ZqExVmcUUAAAAAEDh8Ehjk5eVJkvr06fPjiW0/ntrhcNQ65pZbbpFpmpo7d64vSgBq2Z5fqmOOKknMLgAAAACAxvJJYOAOBNq1a+cZi46O9rzPz8+vdUxKSookafv27b4oAaglO6dG/4IU+hcAAAAAQGP4JDBo27atJKm4uNgz1r59e4WEhEiSNm/eXOsY96yEkpISX5QA1MIKCQAAAABw+nwSGLgfRdiyZYtnLCwszDNe12MHr732miSpY8eOvigBqMW9QoJhSGnMMAAAAACARvFJYHDRRRfJNE0tW7bMa3zMmDEyTVP//ve/NXXqVG3atElff/217rnnHs2bN0+GYejyyy/3RQmAlwpnlbbsr57x0r1ttKLD62/GCQAAAAD4kWGapnmmJ9m0aZPS0tIUHR2t3NxcxcbGSpLKysrUt29f7dq1S4ZheB1jmqbatGmj9evXe/oZtETuWRibNm2yuJLgsj6nUNe++KUk6foByXrmpn7WFgQAAAAAfqKh96E+eyRh2bJlWrBggZxOp2c8MjJSy5Yt05AhQ2Sapterb9++WrJkSYsOC9B0smv0L2CFBAAAAABoPJ/N0x42bFid46mpqfriiy/0/fffa9OmTXI6nerRo4f69+/vq0sDtWSxQgIAAAAAnJFme7C7Z8+e6tmzZ3NdDi2ce4aB3Waod4dYa4sBAAAAgADkk0cSAH9SWuHU9vxSSVKvDjGKCA2xuCIAAAAACDwEBgg6G/cWyd3KM53+BQAAAABwWggMEHS8Gx7SvwAAAAAATgeBAYJOVm7Nhofx1hUCAAAAAAGMwABBxz3DICLUph7toq0tBgAAAAACFIEBgsrho5XKOXxMktS3Y5zsIfyKAwAAAMDp4G4KQaVm/wIeRwAAAACA00dggKCSXaN/QUYnGh4CAAAAwOkiMEBQYYYBAAAAAPgGgQGChmmanhUSYiLs6pIQaXFFAAAAABC4CAwQNPYXlyu/pEKSlJ4SJ8MwLK4IAAAAAAIXgQGCRlbOj/0LeBwBAAAAAM4MgQGCRs3+BRkpNDwEAAAAgDNBYICgUXOFBGYYAAAAAMCZITBAUDBN0zPDIDE6XB3iIqwtCAAAAAACHIEBgsLuQ2UqLndKqn4cgYaHAAAAAHBmCAwQFLJq9C/gcQQAAAAAOHMEBggKXv0LOtHwEAAAAADOFIEBgoL3CgnxltUBAAAAAMGCwAABz1nl0sa9xZKklNat1CYqzOKKAAAAACDwERgg4G3PL9UxR5UkZhcAAAAAgK8QGCDgZefU6F+QQv8CAAAAAPAFAgMEPFZIAAAAAADfIzBAwHOvkGAYUhozDAAAAADAJwI6MFi7dq2uuOIKxcfHKyoqSoMGDdK8efMafZ6DBw/qgQceUI8ePRQREaGEhAQNHjxYL7/8cq19DcM46WvChAk++FRojApnlbbsr2542L1ttKLD7RZXBAAAAADBIWDvrpYtW6bMzExFRERo7NixiomJ0fz58zVmzBjl5ORo0qRJDTrP+vXrNWrUKB05ckQ//elPNXr0aJWWlmrz5s16//33dffdd9c6JjU1tc5woF+/fmf4qdBYm/NK5KgyJdG/AAAAAAB8KSADA6fTqTvvvFM2m02ff/6550b90Ucf1cCBAzVlyhSNHj1aqamppzxPcXGxrrnmGknSt99+q/T09FrXqUuXLl302GOPnfHnwJnLrtG/gBUSAAAAAMB3AvKRhKVLl2rHjh0aN26c17f6cXFxmjJliiorKzVnzpx6z/PSSy9pz549euqpp2qFBZJktwdkntKiZLFCAgAAAAA0iYC8I16+fLkkadSoUbW2ZWZmSpJWrFhR73nmzp0rwzB0ww036Pvvv9fixYt17Ngx9erVS5dddpnCwsLqPK6wsFD//Oc/VVBQoDZt2mjIkCFKS0s7/Q+E0+aeYWC3GerdIdbaYgAAAAAgiARkYLBt2zZJUo8ePWptS0pKUnR0tGefk6msrNSGDRvUtm1bPf/885o6dapcLpdne7du3bRw4cI6g4CsrCzdddddXmOXXXaZ5syZo3bt2p3OR8JpKK1want+qSSpV4cYRYSGWFwRAAAAAASPgHwkoaioehp6XFzdU9BjY2M9+5zM4cOHVVVVpUOHDulPf/qT/vrXv+rAgQPKzc3VH//4R/3www+66qqrVF5e7nXcpEmTtGrVKhUUFKi4uFirVq3S5Zdfrk8++URXXnmlqqqqTnndiooKFRcXe14ul0umaTbi08Nt494iuf/TpdO/AAAAAAB8KiADA19wzyaoqqrSPffco0mTJqldu3ZKTk7Wn/70J914443avXu33n77ba/jZsyYocGDByshIUExMTEaPHiwPvjgAw0bNkxr167Vu+++e8rrPvnkk4qLi/O8tmzZooKCgib7nMHMu+Eh/QsAAAAAwJcCMjBwzyw42SyC4uLik84+OPEcknT11VfX2u4e++abb+qtx2az6c4775Qkffnll6fcd/LkySoqKvK8evXqpcTExHqvgdqycms2PIy3rhAAAAAACEIBGRi4exfU1adg//79Ki0trbO/QU1RUVFKTk6WJMXHx9fa7h47duxYg2py3/QfPXr0lPuFh4crNjbW87LZbDIMo0HXgDf3DIOIUJt6tIu2thgAAAAACDIBGRgMGzZMkrR48eJa2xYtWuS1z6n85Cc/kSR99913tba5x7p06dKgmr766qtG7Y8zc/hopXIOV4c5fTvGyR4SkL/KAAAAAOC3AvIua8SIEerWrZveeOMNrV+/3jNeVFSkadOmKSwsTOPHj/eM5+XlacuWLbUeYfjVr34lSXrqqadUWFjoGd+/f7/+/ve/y2az6YYbbvCMb9iwQQ6Ho1Y9q1at0vTp0xUaGqobb7zRR58Sp1KzfwGPIwAAAACA7wXksop2u10zZ85UZmamLr74Yo0dO1YxMTGaP3++du/erRkzZnh90z958mTNmTNHs2bN0oQJEzzjF154oR588EE988wzSk9P11VXXSWHw6F3331XBw8e1LRp03T22Wd79n/66af14YcfaujQoerUqZNCQ0O1adMmLV68WIZh6MUXX1T37t2b8b9Ey5Vdo39BRicaHgIAAACArwVkYCBJw4cP18qVKzV16lTNnTtXDodDaWlpmj59usaMGdPg8zz99NNKS0vTiy++qNmzZ8swDPXv31+vvPKKrrvuOq99r7nmGhUWFiorK0uffvqpKisrlZSUpLFjx+r+++/XwIEDff0xcRLMMAAAAACApmWYpnsle1ihT58+kqRNmzZZXEngME1TA6ctUX5JhWIj7MqaOorGkQAAAADQQA29Dw3IHgZo2fYXlyu/pEJS9ewCwgIAAAAA8D0CAwScrJwf+xekp9C/AAAAAACaAoEBAg79CwAAAACg6REYIOCwQgIAAAAAND0CAwQU0zQ9MwzaxoQrKTbC2oIAAAAAIEgRGCCg7DpUpuJypyQpIyWOhocAAAAA0EQIDBBQ6F8AAAAAAM2DwAABhRUSAAAAAKB5EBggoDDDAAAAAACaB4EBAoazyqWN+6pnGHRq00ptosIsrggAAAAAgheBAQLGtoOlKne4JDG7AAAAAACaGoEBAkbNxxEy6F8AAAAAAE2KwAABIyu3ZsPDeOsKAQAAAIAWgMAAAcM9w8AwpL7JzDAAAAAAgKZEYICAUO6o0pa8EknSWW2jFR1ut7giAAAAAAhuBAYICJvziuV0mZJ4HAEAAAAAmgOBAQJCdo3+BRmdeBwBAAAAAJoagQECQlaNFRKYYQAAAAAATY/AAAHBPcMgNMRQ7w4xFlcDAAAAAMGPwAB+r7TCqR35pZKknkkxCreHWFwRAAAAAAQ/AgP4vQ25RTKr+x3yOAIAAAAANBMCA/i9DXsLPe8zUmh4CAAAAADNgcAAfi+rxgoJzDAAAAAAgOZBYAC/l318hYSIUJt6tIu2thgAAAAAaCEIDODXDh+tVM7hY5Kkvh3jZA/hVxYAAAAAmgN3X/Br7tkFEo8jAAAAAEBzIjCAX8uu0b8goxMNDwEAAACguRAYwK8xwwAAAAAArEFgAL9lmqZnhYTYCLu6JERaXBEAAAAAtBwEBvBb+4vLlV9SIal6doFhGBZXBAAAAAAtB4EB/FZWzo/9C9JT6F8AAAAAAM2JwAB+i/4FAAAAAGAdAgP4LVZIAAAAAADrEBjAL5mm6Zlh0DYmXEmxEdYWBAAAAAAtDIEB/NKuQ2UqLndKkjJS4mh4CAAAAADNjMAAfon+BQAAAABgLQID+CVWSAAAAAAAaxEYwC8xwwAAAAAArEVgAL/jrHJp477qGQad2rRSm6gwiysCAAAAgJaHwAB+Z9vBUpU7XJKYXQAAAAAAViEwgN+p+ThCBv0LAAAAAMASBAbwO1m5NRsexltXCAAAAAC0YAQG8DvuGQaGIfVNZoYBAAAAAFiBwAB+pdxRpS15JZKks9pGKzrcbnFFAAAAANAyERjAr2zOK5bTZUricQQAAAAAsBKBAfxKdo3+BRmdeBwBAAAAAKxCYAC/klVjhQRmGAAAAACAdQgM4FfcMwxCQwz17hBjcTUAAAAA0HIRGMBvlFY4tSO/VJLUKylW4fYQiysCAAAAgJaLwAB+Y0NukczqfodKT6F/AQAAAABYicAAfiO7Rv+CDPoXAAAAAIClCAzgN2qukJDOCgkAAAAAYCkCA/gN9woJrUJDdFbbaGuLAQAAAIAWjsAAfuFQaYVyjxyTJPVNjpU9hF9NAAAAALASd2XwC9l7azyOQP8CAAAAALAcgQH8QnZOzcCA/gUAAAAAYDUCA/gFVkgAAAAAAP9CYADLmaaprOMrJMS1ClVqQqTFFQEAAAAACAxgubyichWUVkiqfhzBMAyLKwIAAAAAEBjAcjUfR6B/AQAAAAD4BwIDWM79OILECgkAAAAA4C8IDGA5Gh4CAAAAgP8hMIClXC5T2cdnGLSLCVdSXITFFQEAAAAAJAIDWGzXoaMqKXdK4nEEAAAAAPAnAR0YrF27VldccYXi4+MVFRWlQYMGad68eY0+z8GDB/XAAw+oR48eioiIUEJCggYPHqyXX365zv0XLVqkYcOGKSYmRrGxsRo+fLiWLFlyph+nRcqu0b8gg4aHAAAAAOA37FYXcLqWLVumzMxMRUREaOzYsYqJidH8+fM1ZswY5eTkaNKkSQ06z/r16zVq1CgdOXJEP/3pTzV69GiVlpZq8+bNev/993X33Xd77f/666/rlltuUdu2bTVhwgRJ0ty5c3XppZdq3rx5Gj16tK8/alDLqrlCQqd4y+oAAAAAAHgzTNM0rS6isZxOp3r16qXc3FytWbNG/fr1kyQVFRVp4MCB2rVrl7Zu3arU1NRTnqe4uFhpaWk6duyYPvvsM6Wnp9e6jt3+Y6Zy5MgRdevWTXa7XevWrVNKSookKTc3V/3795ck7dy5UzExMQ3+LH369JEkbdq0qcHHBJPRL6/SN7uPSJLW/fFStY4Ks7giAAAAAAhuDb0PDchHEpYuXaodO3Zo3LhxnrBAkuLi4jRlyhRVVlZqzpw59Z7npZde0p49e/TUU0/VCgskeYUFkvTWW2+psLBQEydO9IQFkpSSkqJ7771XBQUFWrBgwel/sBbGWeXSxn3VjyR0bhNJWAAAAAAAfiQgA4Ply5dLkkaNGlVrW2ZmpiRpxYoV9Z5n7ty5MgxDN9xwg77//ns9//zz+utf/6r33ntPlZWVTXZdVNt2sFTlDpckKY3+BQAAAADgVwKyh8G2bdskST169Ki1LSkpSdHR0Z59TqayslIbNmxQ27Zt9fzzz2vq1KlyuVye7d26ddPChQuVlpbWoOu6x+q7bkVFhSoqKjw/u1wuGYZxymOCVXaN/gU0PAQAAAAA/xKQMwyKiqqnscfF1X2TGRsb69nnZA4fPqyqqiodOnRIf/rTn/TXv/5VBw4cUG5urv74xz/qhx9+0FVXXaXy8vIGXTc2NtZrn5N58sknFRcX53lt2bJFBQUFpzwmWGXVWCGBJRUBAAAAwL8EZGDgC+7ZBFVVVbrnnns0adIktWvXTsnJyfrTn/6kG2+8Ubt379bbb7/t0+tOnjxZRUVFnlevXr2UmJjo02sECvcMA8OQ+iYzwwAAAAAA/ElABgbub/hP9m1+cXHxSWcfnHgOSbr66qtrbXePffPNNw26bnFxca3z1iU8PFyxsbGel81ma5GPJJQ7qrQlr0SSdFbbaEWHB+TTMQAAAAAQtAIyMDhVv4D9+/ertLS0zj4DNUVFRSk5OVmSFB8fX2u7e+zYsWMNuu6p+hugts15xXK6qlf05HEEAAAAAPA/ARkYDBs2TJK0ePHiWtsWLVrktc+p/OQnP5Ekfffdd7W2uce6dOni8+tCyq7RvyCjE48jAAAAAIC/MUzTNK0uorGcTqd69uypvXv3as2aNerXr5+k6kcFBg4cqF27dun777/33Ozn5eWpqKhIHTp08HpkYNWqVRoyZIj69OmjlStXemYV7N+/X+edd57y8vK0efNmnX322ZKkI0eOqGvXrgoNDdW6deuUkpIiScrNzVX//v0lSTt37lRMTEyDP0ufPn0kSZs2bTqT/yQB58F56/XO//ZKkhb+eoj6dYq3tiAAAAAAaCEaeh8akDMM7Ha7Zs6cKZfLpYsvvli//OUvNWnSJGVkZGjr1q2aNm2a18yAyZMnq3fv3lqwYIHXeS688EI9+OCD2rRpk9LT0/XrX/9av/zlL5WRkaG9e/fqz3/+sycskKTWrVvrhRdeUEFBgQYMGKCJEydq4sSJGjBggA4dOqSXXnqpUWFBS+aeYRAaYqh3B/6bAQAAAIC/CdhOc8OHD9fKlSs1depUzZ07Vw6HQ2lpaZo+fbrGjBnT4PM8/fTTSktL04svvqjZs2fLMAz1799fr7zyiq677rpa+998881KTEzUtGnTNGvWLBmGoXPPPVePPPKIRo4c6cuPGLRKK5zakV8qSeqVFKtwe4jFFQEAAAAAThSQjyQEk5b4SMLqHYf0s/9bI0n6+QWd9Zfr0iyuCAAAAABajqB+JAGBLTu30PM+gxUSAAAAAMAvERig2dVcISGdFRIAAAAAwC8RGKDZZR2fYdAqNERntY22thgAAAAAQJ0IDNCsDpVWKPfIMUlS3+RY2UP4FQQAAAAAf8TdGppV9t4ajyPQvwAAAAAA/BaBAZpVdk7NwID+BQAAAADgrwgM0KxYIQEAAAAAAgOBAZqNaZrKOr5CQlyrUKUmRFpcEQAAAADgZAgM0GzyispVUFohqfpxBMMwLK4IAAAAAHAyBAZoNjUfR6B/AQAAAAD4NwIDNBv34wgSKyQAAAAAgL8jMECzoeEhAAAAAAQOAgM0C5fLVPbxGQbtYsKVFBdhcUUAAAAAgFMhMECz2HXoqErKnZJ4HAEAAAAAAgGBAZpFdo3+BRk0PAQAAAAAv0dggGaRVXOFhE7xltUBAAAAAGgYAgM0i5ozDNKTmWEAAAAAAP6OwABNzlnl0qZ91YFB5zaRah0VZnFFAAAAAID6EBigyW09UKpyh0uSlE7/AgAAAAAICAQGaHLZNfoXZLBCAgAAAAAEBAIDNLmsmv0LmGEAAAAAAAGBwABNzj3DwGZIfWl4CAAAAAABgcAATarcUaXv95dIks5qF62ocLvFFQEAAAAAGoLAAE3qu7xiOV2mJCmd/gUAAAAAEDAIDNCksnMKPe8z6F8AAAAAAAGDwABNKtur4WG8dYUAAAAAABqFwABNKut4w8PQEEO9OsRYWwwAAAAAoMEIDNBkSsod2llwVJLUu0Oswu0hFlcEAAAAAGgoAgM0mQ17i2RW9ztUOv0LAAAAACCgEBigydC/AAAAAAACF4EBmkz28f4FkpRBYAAAAAAAAYXAAE0mK6d6hkFkWIjOahdtcTUAAAAAgMYgMECTOFRaob2FxyRJfTvGKcRmWFwRAAAAAKAxCAzQJLL31uxfQMNDAAAAAAg0BAZoEtk5NQKDTvHWFQIAAAAAOC0EBmgS3g0PmWEAAAAAAIGGwAA+Z5qmso4vqRgfGarObSItrggAAAAA0FgEBvC5vKJyFZRWSJLSkuNkGDQ8BAAAAIBAQ2AAn6v5OAINDwEAAAAgMBEYwOfcjyNIUnpKvHWFAAAAAABOG4EBfM674WG8ZXUAAAAAAE4fgQF8yuUylX18hkG7mHAlxUVYXBEAAAAA4HQQGMCndh06qpJypyQeRwAAAACAQEZgAJ/KrtG/IIOGhwAAAAAQsAgM4FNZNVdI6BRvWR0AAAAAgDNDYACfqjnDID2ZGQYAAAAAEKgIDOAzziqXNu2rDgw6t4lU66gwiysCAAAAAJwuAgP4zNYDpSp3uCRJ6fQvAAAAAICARmAAn8mu0b8ggxUSAAAAACCgERjAZ7Jq9i9ghgEAAAAABDQCA/iMe4aBzZD60vAQAAAAAAIagQF8otxRpe/3l0iSzmoXrahwu8UVAQAAAADOBIEBfOK7vGI5XaYkKZ3+BQAAAAAQ8AgM4BPZOYWe9xn0LwAAAACAgEdgAJ/I9mp4GG9dIQAAAAAAnyAwgE9kHW94GBpiqFeHGGuLAQAAAACcMQIDnLGScod2FhyVJPXuEKtwe4jFFQEAAAAAzhSBAc7Yhr1FMqv7HSqd/gUAAAAAEBQIDHDG6F8AAAAAAMGHwABnLPt4/wJJyiAwAAAAAICgQGCAM5aVUz3DIDIsRGe1i7a4GgAAAACALxAY4IwcKq3Q3sJjkqS+HeMUYjMsrggAAAAA4AsEBjgj3v0LaHgIAAAAAMGCwABnJKtG/4L0TvGW1QEAAAAA8K2ADgzWrl2rK664QvHx8YqKitKgQYM0b968Bh8/e/ZsGYZx0tfy5ctrHdOlS5eT7n/JJZf47sMFiJozDDKYYQAAAAAAQcNudQGna9myZcrMzFRERITGjh2rmJgYzZ8/X2PGjFFOTo4mTZrU4HNdc8016tevX63xLl261Ll/XFyc7r///gbvH6xM0/SskBAfGarObSKtLQgAAAAA4DMBGRg4nU7deeedstls+vzzzz03+48++qgGDhyoKVOmaPTo0UpNTW3Q+a699lpNmDChwdePj4/XY4891vjCg8y+onIVlFZKktKS42QYNDwEAAAAgGARkI8kLF26VDt27NC4ceO8ZgbExcVpypQpqqys1Jw5c6wrsIXIzin0vM9IibesDgAAAACA7wXkDAN3b4FRo0bV2paZmSlJWrFiRYPPt27dOh06dEhOp1NdunTRyJEjlZCQcNL9KyoqNHv2bO3bt0+xsbE6//zzdcEFFzTuQwSBLFZIAAAAAICgFZCBwbZt2yRJPXr0qLUtKSlJ0dHRnn0a4h//+IfXz61atdLUqVP1+9//vs799+/fr9tuu81r7Pzzz9d///tfde/e/ZTXqqioUEVFhednl8sVsFP5s2uskJDBCgkAAAAAEFQC8pGEoqLqb7bj4ur+Vjs2Ntazz6l07dpVzz//vLZu3aqysjLl5ubq1VdfVZs2bfTwww/r+eefr3XMbbfdpiVLlujAgQM6evSo1q1bp1tuuUVr167ViBEjVFJScsprPvnkk4qLi/O8tmzZooKCggZ8av/icpnacHyGQfvYcLWPjbC4IgAAAACALxmmaZpWF9FYo0aN0qeffqpt27bprLPOqrU9OTlZpaWlDQoN6rJp0yadd955ioyM1IEDB2S31z8RY/z48Xrttdf09NNP68EHHzzpfifOMLjgggtkGIa+++6706rVKjvySzXi6erHPi49p73+b/x5FlcEAAAAAGiIPn36SKq+9z2VgJxh4J5ZcLJAoLi4+KSzDxqiT58+Gjp0qA4fPqzNmzc36Ji77rpLkvTll1+ecr/w8HDFxsZ6XjabLSAfSfB6HIH+BQAAAAAQdAIyMHD3LqirT8H+/ftVWlpaZ3+DxkhMTJQkHT16tEn2D3RZOTUbHsZbVwgAAAAAoEkEZGAwbNgwSdLixYtrbVu0aJHXPqejqqpK33zzjSQpNTW1Qcd89dVXkqQuXbqc9nUDSc0ZBqyQAAAAAADBJyADgxEjRqhbt2564403tH79es94UVGRpk2bprCwMI0fP94znpeXpy1bttR6hOHbb7+tde6qqio9/PDD2r59u4YPH64OHTp4tm3ZskVlZWW1jtmyZYtnRYVx48ad6cfze44qlzbtK5YkpSZEKj4yzOKKAAAAAAC+FpDLKtrtds2cOVOZmZm6+OKLNXbsWMXExGj+/PnavXu3ZsyY4fVN/+TJkzVnzhzNmjVLEyZM8Iyfd955Sk9PV3p6upKTk3X48GGtWLFCW7duVUpKimbOnOl13TfffFPPPPOMLr74YqWmpioqKkpbt27VRx99JIfDocmTJ+viiy9upv8K1tl6oEQVTpckHkcAAAAAgGAVkIGBJA0fPlwrV67U1KlTNXfuXDkcDqWlpWn69OkaM2ZMg84xadIkrVmzRp9++qkOHz6ssLAwnXXWWXrkkUf04IMPqnXr1rWuuXnzZq1bt05ffPGFysrKlJiYqCuuuEL33HOPRo0a1RQf1e+4l1OUaHgIAAAAAMEqIJdVDCYNXc7Cn0x+Z4P++/UeSdK8uwZrYNc2FlcEAAAAAGiooF5WEdZyNzy0GVLf5FhriwEAAAAANAkCAzRKuaNK3+8vkST1aBejyLCAfaoFAAAAAHAKBAZolO/yiuV0VT/FwnKKAAAAABC8CAzQKNk5hZ736Z3iLasDAAAAANC0CAzQKNmskAAAAAAALQKBARol63jDw7AQm3ol0fAQAAAAAIIVgQEarKTcoZ0FRyVJvTvEKMzOrw8AAAAABCvu+NBgG/YWyazud6g0HkcAAAAAgKBGYIAGq9m/ID0l3rpCAAAAAABNjsAADZZ9vH+BJGUQGAAAAABAUCMwQINl5VTPMIgMC9FZ7aItrgYAAAAA0JQIDNAgh0ortLfwmCSpb8c4hdgMiysCAAAAADQlAgM0iHf/AhoeAgAAAECwIzBAg2TV6F+Q3inesjoAAAAAAM2DwAANUnOGQQYzDAAAAAAg6BEYoF6maXpWSIiPDFXnNpHWFgQAAAAAaHIEBqjXvqJyFZRWSpLSkuNkGDQ8BAAAAIBgR2CAemXnFHreZ6TEW1YHAAAAAKD5EBigXnsLj8m9iiIrJAAAAABAy2C3ugD4vzsu6qZxF3TWpn3F6pkUY3U5AAAAAIBmQGCABokMs+v8Lm2sLgMAAAAA0Ex4JAEAAAAAANRCYAAAAAAAAGohMAAAAAAAALUQGAAAAAAAgFoIDAAAAAAAQC0EBgAAAAAAoBYCAwAAAAAAUAuBAQAAAAAAqIXAAAAAAAAA1EJgAAAAAAAAaiEwAAAAAAAAtRAYAAAAAACAWggMAAAAAABALQQGAAAAAACgFgIDAAAAAABQC4EBAAAAAACohcAAAAAAAADUQmAAAAAAAABqITAAAAAAAAC1EBgAAAAAAIBaCAwAAAAAAEAtBAYAAAAAAKAWAgMAAAAAAFALgQEAAAAAAKjFME3TtLqIliwmJkYOh0Pdu3e3uhQAAAAAQAuwY8cOhYaGqqSk5JT7McPAYlFRUQoNDbW6DPgp0zSVn58vcj20BPy+oyXh9x0tCb/vaEkC5fc9NDRUUVFR9e7HDAPAjxUXFysuLk5FRUWKjY21uhygSfH7jpaE33e0JPy+oyUJtt93ZhgAAAAAAIBaCAwAAAAAAEAtBAYAAAAAAKAWAgPAj4WHh2vq1KkKDw+3uhSgyfH7jpaE33e0JPy+oyUJtt93mh4CAAAAAIBamGEAAAAAAABqITAAAAAAAAC1EBgAAAAAAIBaCAwAAAAAAEAtBAaAH9m7d6+ee+45jRo1Sp07d1ZYWJiSkpJ0ww036KuvvrK6PKBZTJ8+XYZhyDAMrVmzxupyAJ9bsGCBLr30UiUkJCgiIkJdu3bVz372M+Xk5FhdGuAzpmnqnXfe0fDhw9WhQwdFRkaqZ8+euuuuu7Rz506rywMa7fXXX9ddd92l8847T+Hh4TIMQ7Nnzz7p/sXFxXrwwQeVmpqq8PBwdenSRQ899JBKS0ubr2gfYJUEwI88/PDDmj59urp3765LLrlEbdu21bZt27Rw4UKZpqk33nhDY8aMsbpMoMls3LhR5513nux2u44eParVq1dr0KBBVpcF+IRpmvrVr36lf/7zn+revbsyMzMVExOjffv2acWKFfrPf/6joUOHWl0m4BOTJk3SM888ow4dOuiaa65RbGyssrKytHjxYkVHR2vVqlXq27ev1WUCDdalSxft3r1biYmJioqK0u7duzVr1ixNmDCh1r5Hjx7V0KFDtX79eo0aNUr9+/fXunXrtHjxYp1//vn6/PPPFRER0fwf4nSYAPzG/PnzzeXLl9ca//zzz83Q0FCzdevWZnl5uQWVAU2vsrLSHDBggHnBBReYN998synJXL16tdVlAT7z3HPPmZLMe+65x3Q6nbW2OxwOC6oCfC8vL8+02WxmamqqWVhY6LXtmWeeMSWZt912m0XVAafn008/NXft2mWapmk++eSTpiRz1qxZde776KOPmpLM3//+917jv//9701J5rRp05q6XJ/hkQTAj1x//fUaNmxYrfGLLrpIw4cP15EjR7RhwwYLKgOa3l/+8hdt2rRJ//73vxUSEmJ1OYBPHTt2TI8//ri6deumv//973X+jtvtdgsqA3xv165dcrlcGjJkiOLi4ry2XXnllZKk/Px8K0oDTtvIkSOVmppa736maWrmzJmKjo7WH//4R69tf/zjHxUdHa2ZM2c2VZk+R2AABIjQ0FBJ/IMSwel///uf/vKXv2jq1Kk655xzrC4H8LnFixfryJEjuvbaa1VVVaV33nlHTz31lF555RVt377d6vIAn+rRo4fCwsL05Zdfqri42GvbBx98IEkaMWKEFaUBTW7btm3at2+fhgwZoqioKK9tUVFRGjJkiHbu3BkwfWu48wACwJ49e/TZZ5+pQ4cOSktLs7ocwKcqKio0fvx49evXT7/73e+sLgdoEt9++60kKSQkROnp6dq6datnm81m0wMPPKAZM2ZYVR7gUwkJCXrqqac0adIk9erVy6uHwdKlS3XPPffo3nvvtbpMoEls27ZNUnVwVpcePXpo0aJF2rZtmzp16tScpZ0WAgPAzzkcDt1yyy2qqKjQ9OnTmaqNoPPoo49q27Zt+vbbb/n9RtA6ePCgJOmZZ57RgAED9PXXX6t3795at26dfvnLX+rpp59W9+7ddffdd1tcKeAbDzzwgJKTk3XHHXfolVde8YwPHTpU48aNY8YkglZRUZEk1Xocxy02NtZrP3/HIwmAH3O5XJowYYI+//xz3XnnnbrlllusLgnwqdWrV2vGjBl65JFH6JaNoOZyuSRJYWFhWrhwoc4//3xFR0froosu0ltvvSWbzaann37a4ioB3/nTn/6km2++WVOmTFFOTo5KSkr0xRdfqLy8XJdcconee+89q0sE0AAEBoCfcrlc+sUvfqE33nhDN998s1c6DwQDp9OpW2+9Venp6Xr44YetLgdoUu5vms477zx17NjRa1vfvn3VrVs37dixQ4WFhRZUB/jWZ599pqlTp+ree+/Vww8/rJSUFEVHR2vo0KF6//33FRoaqkmTJlldJtAk3H/fn2wGgbuvx8lmIPgb5gIBfsjlcum2227Tq6++qp/97GeaPXu2bDbyPQSX0tJSz3N+YWFhde4zePBgSdKCBQt07bXXNldpgM/17NlTkhQfH1/ndvf4sWPHTroPECg+/vhjSdLw4cNrbUtKSlKvXr20bt06lZaWKjo6urnLA5qUu3eB+984J6qvx4G/ITAA/EzNsGDMmDF67bXXeK4bQSk8PFy33357nds+//xzbdu2TVdffbXatm2rLl26NG9xgI+5b5w2b95ca5vD4dD27dsVFRWltm3bNndpgM9VVlZKOvnSifn5+bLZbJ4VoIBg0qNHD3Xs2FFffvmljh496rVSwtGjR/Xll1+qa9euAdHwUOKRBMCvuB9DePXVV3XjjTfq9ddfJyxA0GrVqpVmzpxZ5+vCCy+UJE2ePFkzZ85Uv379rC0WOEPdu3fXqFGjtH379lrrbz/11FMqLCzUddddRyM4BIUhQ4ZIqm7yeeK07FdeeUW5ubkaPHiwwsPDrSgPaFKGYeiOO+5QaWmpnnjiCa9tTzzxhEpLS3XnnXdaVF3jGaZpmlYXAaDaY489pscff1zR0dG677776vyH47XXXsvNE4LehAkTNGfOHK1evVqDBg2yuhzAJ3bs2KELL7xQBw8e1E9/+lPPtOylS5cqNTVVa9asUVJSktVlAmesqqrq/7d3p0FZlW8cx38giIK4QKwN5ZbGmCSayPRgOFYGGmY5krk3mcaQNi4vykR0JnVG/wqlo+aGlIqglrGU4sIikMomOZmjpow6iikq45KieP4vHJjwgUcQlcjv5xWe+zrXue7DC8br3Oc+6t+/vzIzM+Xq6qrBgwerbdu2Kigo0J49e9SyZUulp6fLz8+vsUsF6mz16tXKysqSJB06dEgFBQUymUzq3LmzpHtfABk/frykeysJTCaTioqKNGDAAPXs2VMFBQVKTU1V7969lZGRoZYtWzbaXOqDNjbwL1JcXCzp3rvdc+fOrTGmffv2NAwAoAnq1KmT8vLyNGvWLG3fvl2pqalyd3dXeHi4Zs2aJVdX18YuEXgkmjVrptTUVEVFRSkhIUEbN25UeXm53Nzcqr6c4O3t3dhlAvWSlZWl2NjYaseys7OVnZ1d9e/KhoGDg4MyMjI0e/Zsbd26VWlpafLw8NC0adMUGRnZZJoFEisMAAAAAABADdjDAAAAAAAAmKFhAAAAAAAAzNAwAAAAAAAAZmgYAAAAAAAAMzQMAAAAAACAGRoGAAAAAADADA0DAAAAAABghoYBAAAAAAAwQ8MAAACgka1bt05WVlaysrJScXFxY5cDAIAkGgYAAAAAAKAGNAwAAAAAAIAZK8MwjMYuAgAAAAAA/LuwwgAAAAAAAJihYQAAAAAAAMzQMAAA4AmZPXt21U74knTz5k0tXLhQPXv2lKOjoxwdHeXn56elS5fqzp07teapzDF79myL1+vXr5+srKzUr18/s7H09PSqPOnp6TIMQ2vWrFFAQICcnZ3VunVr+fn56fvvv692Xnl5uVasWCF/f385OTnJ0dFRJpNJCQkJ9b4fNamoqNCyZcvUp08ftW7dWm3atFHPnj31v//9T7du3VJxcXFV3evWravXnP/p/t9FbcrKyjR//nyZTCa5uLioefPm8vDwUEhIiLZs2SJLb3be/3vas2ePhg0bJi8vL9na2qp9+/ZVsXX9SkJFRYViY2P19ttvy9PTU3Z2dnJ2dlZAQIAWL16sv//+2+J88vPz9dFHH6lLly5ycHBQixYt5OXlpV69eik8PFyJiYkW5wQAeLrYNHYBAAA8jc6fP6+goCAdPHiw2vHc3Fzl5uYqNTVV27Ztk7X14+/t3759W++8846SkpLMahkzZozy8vL09ddf6/LlyxoyZIgyMzOrxeXk5CgnJ0fHjx/XjBkzHrqOa9euaeDAgdq7d2+144WFhSosLFRcXJxWr1790Pnra/fu3Xr//fdVWlpa7XhJSYmSk5OVnJysgQMHKj4+Xq1atbKY68svv9S8efMaVM+pU6c0ePBgFRUVVTt+6dIlZWdnKzs7W8uXL1dKSoq6dOlidn5UVJSmT5+uu3fvVjt+5swZnTlzRgUFBVq2bJmuXr36wPkAAJ4OrDAAAKARvPfeezp8+LAmT56snTt3Kj8/Xxs3bpS3t7ckKSkpSatWrXoitURERCgpKUkjR45USkqK8vPzFRcXp65du0qSvvnmG+3atUvjxo1TTk6OwsLClJqaqvz8fK1Zs0aenp6SpFmzZun3339/6DpGjRpV1Szw8/NTXFyc8vLylJKSomHDhqmgoEATJ05s+ITrIDs7W8HBwSotLZWbm5u++uorJSUlKT8/X0lJSRo1apQk6eeff9bYsWMt5vrhhx80b948de/eXWvXrtWBAweUkZGhqVOn1rme0tJSBQQEqKioSHZ2dvr000+1efNm5ebmKi0tTV988YXs7e11/PhxBQcHq6ysrNr5v/32W1WzoEOHDlq0aJF2796twsJCZWZmatWqVRoxYoQcHBzqf7MAAP9dBgAAeCIiIyMNSYYkw9bW1khLSzOLKS0tNdzc3AxJho+PT415KnNERkZavF5gYKAhyQgMDDQbS0tLq8ojyYiOjjaLOXfunOHo6GhIMlxcXAwrKyvjxx9/NIsrKioyrK2tDUnG5MmTLdZUm+Tk5KpaBg4caNy+fdssZs6cOdVqjomJMYuxNOd/+ufv4n7l5eVG+/btDUlGUFCQcf369RpzrFy5sipHamqq2fg/a3399deNmzdv1lpPTExMVezJkyfNxkeMGGFIMp5//nnjxIkTNeYoKCgwHBwcDEnGjBkzqo1FREQYkgwHBwejpKSk1jquXLliVFRU1DoOAHi6sMIAAIBGMGnSpBrfs3dyctKHH34oSTp06JDZk+LHoU+fPvrss8/Mjru7u+vdd9+VJF24cEGhoaEaMmSIWZyPj48CAgIkyex1grpatmyZJMnOzk6rVq2SjY35W5MzZ87USy+99FD562PTpk0qLi5WixYt9N1338ne3r7GuI8//lh+fn6SVON+CpWsra21evVq2dnZPVQ9xcXFio+PlyQtXbpUHTp0qDHO19dX4eHhNdZTUlIiSerSpYvc3NxqvVabNm2eyGswAICmgb8IAAA0gpEjR9Y61qtXL0mSYRg6efLkY69l+PDhtY69/PLL9Yo7ceJEva9fUVGh9PR0SdKAAQOqXnG4n7W19QOX/z8KiYmJkqTAwEC5uLhYjH3ttdckSb/++mutMSaTqdoGh/WVkpKiiooK2dvbKzg4uE71nD17VqdOnao67uHhIUk6fPiwDhw48NC1AACeLmx6CABAI3jxxRdrHXNycqr6+erVq4+9lpo2yKvUtm3besU9TL1//vmnbty4IUnq3bu3xdjKJ/qPU15eniRpx44dD/yKQqXKJ/g18fHxeST13Lhxo8aVF5Zqeu655yRJH3zwgebPn69bt27JZDIpKChIgwYNUkBAgLp161bneQIAni6sMAAAoBHUtsxdUrUl4RUVFf+aWuoSd/8O/HVx6dKlqp9dXV0txlpaTv+o/PXXX/U+x9LnDNu1a9eQch6qHklVTRjpXoMqLi5O7dq10507d5ScnKywsDB1795drq6uGj169EO/TgIA+O9ihQEAAPjX+Dc86a5s0gQHB2vBggUNztesWbNHUs8zzzyjtLS0Op93/14HQ4cO1RtvvKH4+Hjt2LFDe/fu1YULF3Tx4kWtX79e69ev19ixY7V27Vr2MQAASKJhAABAk2NlZSXDMB74NP/69etPqKKG+ecT+PPnz1uMfdB4XVc6WLo3zs7OOnv2rMrLy5/IJosP4uzsLOne6x7e3t4NakC0adNGEyZM0IQJEyRJf/zxh3766SctWbJEZ8+eVWxsrHx9fWvcBBMA8PShfQwAQBPj6OgoSbp8+XKtMYZh6Pjx40+qpAbp1KmTWrZsKUnKzc21GPug8brcG0k6evRorWO+vr6S7u0dUF5ebjHPk1BZz61bt6r2M3hUvL299fnnn2vfvn1ycHCQJCUkJDzSawAAmi4aBgAANDGVS80t/efxl19+0ZUrV55QRQ1jY2NT9YnJ1NRUnTt3rsa4u3fvKjY21mKuyntz9OjRWjdgvHjxonbu3FlrjsGDB0uSysrKFBMT86DyH7uQkJCqVzWio6MfyzW8vLyqNrW8ePHiY7kGAKDpoWEAAEATExgYKEnav3+/srOzzcZLSko0adKkJ11Wg4SFhUm69xR94sSJNW72OH/+fB06dMhinsp7U15eriVLlpiN3759W+PHj7e4SeHYsWPl5eUlSZo+fboyMzMtXjMrK0sZGRkWYxqia9euGjZsmCRp06ZNWrx4scX4kydPKi4urtqxbdu2WWwgnT59WkeOHJFkvvcBAODpRcMAAIAmZsKECbKxsZFhGAoJCVF0dLTy8vKUk5OjhQsXytfXV2VlZXrhhRcau9Q6CwkJUUhIiCQpKSlJJpNJ8fHxKigo0Pbt2zV8+HDNnDlTr7zyisU8gwYN0vPPPy9JioiI0NSpU5WVlaW8vDzFxsbK399fiYmJ8vf3rzWHnZ2dEhISZGdnp2vXrql///4aNWqUtmzZovz8fOXm5ioxMVGRkZHy8fFR3759H9jIaKjly5erY8eOkqRp06YpMDBQa9as0b59+1RYWKhdu3Zp0aJFevPNN9W5c2dt3bq12vnR0dF69tlnFRoaqhUrVigjI0MHDx5UWlqaFi5cKJPJVNVE+eSTTx7rXAAATQebHgIA0MR069ZNCxYs0NSpU3X58mVNmTKl2riTk5O2bdumiIgIHTt2rJGqrL8NGzYoODhY2dnZ2r9/v4YPH15t3NfXV99++6169epVa47mzZtr/fr1CgoK0vXr1xUVFaWoqKiq8WbNmik6OlqXLl3Svn37as3j7++v9PR0hYaG6vTp09qwYYM2bNhQa3zr1q3rMdP6c3JyUnZ2tkJDQ7V3715lZmZaXPlQUz03btzQ5s2btXnz5hrPsba21pw5czRkyJBHVTYAoIljhQEAAE3QlClTtH37dr311ltq166d7Ozs1KFDB4WHh6uwsFB9+/Zt7BLrzdHRUenp6VqyZIl69+6tVq1aydHRUT169ND8+fOVk5MjJyenB+YJCAhQfn6+Ro8eLU9PT9na2srDw0NDhw5VZmamJk+eXKd6/P39dezYMa1YsUKDBg2Sp6enmjdvrhYtWsjLy0sDBgzQ3LlzdeTIEY0ZM6ah038gd3d3ZWZmKjk5WSNHjlTHjh1lb28vW1tbubi46NVXX9W0adOUkZGhtWvXVjs3Li5OK1eu1IgRI9SjRw+5u7vLxsZGrVq1Urdu3RQWFqbCwkLNnDnzsc8DANB0WBmGYTR2EQAAAHVRXFxc9Y59TEyMxo0b17gFAQDwH8YKAwAAAAAAYIaGAQAAAAAAMEPDAAAAAAAAmKFhAAAAAAAAzNAwAAAAAAAAZvhKAgAAAAAAMMMKAwAAAAAAYIaGAQAAAAAAMEPDAAAAAAAAmKFhAAAAAAAAzNAwAAAAAAAAZmgYAAAAAAAAMzQMAAAAAACAGRoGAAAAAADADA0DAAAAAABg5v/NuSthu/FtWAAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 1200x800 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "fig = plt.figure(figsize=(12, 8))\n",
    "ax = plt.axes()\n",
    "\n",
    "ax.plot(np.arange(num_queries+1), results_setfit, label='SetFit')\n",
    "ax.legend()\n",
    "\n",
    "plt.ylabel('accuracy')\n",
    "plt.xlabel('num queries')\n",
    "\n",
    "display(plt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e0b2d085-e9a6-4df0-9fe4-781dc2e62887",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.18"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}