NNsight_v0_4_guide.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JYWncbOUZzlm"
      },
      "source": [
        "# `nnsight 0.4`: walkthrough\n",
        "**We have many exciting new features in this update, including:**\n",
        "\n",
        "*   Descriptive error messages\n",
        "*   .all() for multiple token generation\n",
        "*   vLLM Integration\n",
        "*   Streaming remote execution to local machine\n",
        "*   Support for traditional Python syntax for `if` statements and `for` loops on proxies within tracing contexts\n",
        "*   Ability to rename model modules\n",
        "\n",
        "...and more!\n",
        "\n",
        "The following walkthrough guides you through how to access `nnsight 0.4` and use all of its individual features. \n",
        "\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "FNqrMwiUmqOB"
      },
      "source": [
        "**Breaking Changes**\n",
        "* The InterventionGraph now follows a <u>sequential execution order</u>. Module envoys are expected to be referenced following the model’s architecture hierarchy. This means that out-of-order in-place operations will not take effect.\n",
        "\n",
        "* Saved node values are automatically injected into their proxy reference in the Python frames post graph execution. If you are calling `.value` in your code after tracing, this could lead to the wrong behavior."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6a2QUOA7URPF"
      },
      "source": [
        "# Access `nnsight` 0.4"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "id": "kq2qmH42sFX9"
      },
      "outputs": [],
      "source": [
        "from IPython.display import clear_output\n",
        "!pip install nnsight\n",
        "clear_output()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6SeRg2jrkpt0"
      },
      "source": [
        "Import `nnsight` and load the GPT-2 model."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "id": "qakQzKv8kRQh"
      },
      "outputs": [],
      "source": [
        "# import packages\n",
        "import nnsight\n",
        "from nnsight import LanguageModel"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 713,
          "referenced_widgets": [
            "0f3eaadfc2c34384afd2979417d94812",
            "a2eafc8f9262412793f35b372eb300eb",
            "cb3a1a3578554cac9cec37d9b0390c1a",
            "17f30f75790e4273b3e475f433c4760d",
            "1046b0563cb64ce68fc7b1b2f5d3d65d",
            "886f12cb69c2435d978eb2d33534763a",
            "cb0ab1cce5364b44ba85e65ccf44d0c9",
            "0650393001b34123a0e18663e6aa0fcb",
            "cdfa2b53fdd14d24b9ccc0920f34bd69",
            "2e14c72eaf6644bd9d656513c302c278",
            "30af0b2c974d418cb40b30403a2a90cf",
            "c3b4c47b73d544eaba67d5828f1f742d",
            "c924f41345d34f72a00ed3b21f84de82",
            "0e4b52697a87475882e40b3ca4a2e77d",
            "c4c1c2ccc4014520bb265d21550fb133",
            "3160c45575ad4426b2abcfaccd27d41c",
            "4eb7af991ebe4bdd855d3cdd50184495",
            "c96d870993144ed09e54de5dc00fad6d",
            "60627b23fe934d01b4a4af1f592fbd55",
            "a806062fe502459a93ee212e57af2e66",
            "9b161849be8a4464a486e6270f3c0f3b",
            "f2ca144c6f8d4537ac8e2e29c8109454",
            "33c1fe5504d1441e805a02fb4693acd7",
            "8bf1249ac7aa46b094bc899d432153fa",
            "c60da3ab273d4439a0baec6ae7c216e4",
            "9ef4dc73855c4fa69656f75110ac390d",
            "4b1d8724a6b247e0a3dafe2f6b9de178",
            "6909cf23963d4347820b3717b62b1cfe",
            "f6a764157eab4ee7b24ce3dc3e317222",
            "59f8564845764c9e9891274216be83ff",
            "3adee7a17b2a4f50b9ff36fe10a3b701",
            "e60f4ecadf75463ca5fc5f934954e144",
            "d5f42dda1612435f8eb053fe29f711c0",
            "a1595afd22fa4d999a651e1ff63f4279",
            "5c5d0fc4f29141fc95008070c0b077d3",
            "c410466db4ac4d878c3aac7fe4a0df5b",
            "1ac5508e75154bd18275d6f6775242b7",
            "6fc349dcab8d43309faaa08775f445ff",
            "4d7f03cfa9944554b093b3e5c691db92",
            "732fe1ca67b74026805de4688d81e6e3",
            "3c1c03b756c1401da717b3b62842a1f9",
            "416faee108c44dce9e1b3527e82acb9b",
            "a22aff0a7b33470b970a32269a9a0f08",
            "be8e4e0c8e0c482897194f08c77f82a0",
            "d86efefd42444836b476a5caecccdb75",
            "2f851281cd2448f8a36c9b538931e0c9",
            "9a2747e522c94c00af3f9d0bbd54cc1d",
            "176749c3ecd24b829a78f892dee8af91",
            "33cc1f2b3b884a65a4a942cde1662b8f",
            "637e15c89e6847098b27f7a4f3482a63",
            "e039919f976f4d5fbebb5117ec53d57e",
            "7c3d6693887b4c29b3d63afd37e4292e",
            "6386746d591e4e4cb83a18dcc4481678",
            "0325fe95b3e3432497fc68317b2126d4",
            "fabbb16b5e7942f493b7dc92279715fd"
          ]
        },
        "id": "0kXdv3B51oEt",
        "outputId": "e76e7f65-56c0-4244-8e57-98a2475088a4"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "0f3eaadfc2c34384afd2979417d94812",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "config.json:   0%|          | 0.00/665 [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "c3b4c47b73d544eaba67d5828f1f742d",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "tokenizer_config.json:   0%|          | 0.00/26.0 [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "33c1fe5504d1441e805a02fb4693acd7",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "vocab.json:   0%|          | 0.00/1.04M [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "a1595afd22fa4d999a651e1ff63f4279",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "merges.txt:   0%|          | 0.00/456k [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "d86efefd42444836b476a5caecccdb75",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "tokenizer.json:   0%|          | 0.00/1.36M [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "GPT2LMHeadModel(\n",
            "  (transformer): GPT2Model(\n",
            "    (wte): Embedding(50257, 768)\n",
            "    (wpe): Embedding(1024, 768)\n",
            "    (drop): Dropout(p=0.1, inplace=False)\n",
            "    (h): ModuleList(\n",
            "      (0-11): 12 x GPT2Block(\n",
            "        (ln_1): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
            "        (attn): GPT2SdpaAttention(\n",
            "          (c_attn): Conv1D()\n",
            "          (c_proj): Conv1D()\n",
            "          (attn_dropout): Dropout(p=0.1, inplace=False)\n",
            "          (resid_dropout): Dropout(p=0.1, inplace=False)\n",
            "        )\n",
            "        (ln_2): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
            "        (mlp): GPT2MLP(\n",
            "          (c_fc): Conv1D()\n",
            "          (c_proj): Conv1D()\n",
            "          (act): NewGELUActivation()\n",
            "          (dropout): Dropout(p=0.1, inplace=False)\n",
            "        )\n",
            "      )\n",
            "    )\n",
            "    (ln_f): LayerNorm((768,), eps=1e-05, elementwise_affine=True)\n",
            "  )\n",
            "  (lm_head): Linear(in_features=768, out_features=50257, bias=False)\n",
            "  (generator): Generator(\n",
            "    (streamer): Streamer()\n",
            "  )\n",
            ")\n"
          ]
        }
      ],
      "source": [
        "model = LanguageModel('openai-community/gpt2', device_map='auto')\n",
        "print(model)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yVuOTllrUjgv"
      },
      "source": [
        "# Improved Error Messaging"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "45AT75G4UmRk"
      },
      "source": [
        "If you've been using `nnsight`, you are probably familiar with the following type of error message:\n",
        "\n",
        "```\n",
        "IndexError: Above exception when execution Node: 'setitem_0' in Graph: '6063279136'\n",
        "```\n",
        "It can be quite difficult to troubleshoot with these errors, so in `nnsight 0.4` we've now improved error messaging to be descriptive and line-specific! Let's check it out:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 365
        },
        "id": "kUtl8NUqWSbJ",
        "outputId": "0fc43241-7f46-4ab8-831a-089121a68e42"
      },
      "outputs": [
        {
          "ename": "IndexError",
          "evalue": "list index out of range",
          "output_type": "error",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mIndexError\u001b[0m                                Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-4-c0aea694bac2>\u001b[0m in \u001b[0;36m<cell line: 0>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[0mprompt\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m'The Eiffel Tower is in the city of'\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      2\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 3\u001b[0;31m \u001b[0;32mwith\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtrace\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mprompt\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0mtracer\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      4\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      5\u001b[0m     \u001b[0;31m# try to access a layer of model that doesn't exist\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.11/dist-packages/nnsight/intervention/contexts/interleaving.py\u001b[0m in \u001b[0;36m__exit__\u001b[0;34m(self, exc_type, exc_val, exc_tb)\u001b[0m\n\u001b[1;32m     94\u001b[0m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_model\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_envoy\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_reset\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     95\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 96\u001b[0;31m         \u001b[0msuper\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__exit__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mexc_type\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mexc_val\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mexc_tb\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     97\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     98\u001b[0m     \u001b[0;34m@\u001b[0m\u001b[0mclassmethod\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.11/dist-packages/nnsight/tracing/contexts/tracer.py\u001b[0m in \u001b[0;36m__exit__\u001b[0;34m(self, exc_type, exc_val, exc_tb)\u001b[0m\n\u001b[1;32m     23\u001b[0m         \u001b[0mGlobalTracingContext\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtry_deregister\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     24\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 25\u001b[0;31m         \u001b[0;32mreturn\u001b[0m \u001b[0msuper\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m__exit__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mexc_type\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mexc_val\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mexc_tb\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     26\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     27\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0miter\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcollection\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.11/dist-packages/nnsight/tracing/contexts/base.py\u001b[0m in \u001b[0;36m__exit__\u001b[0;34m(self, exc_type, exc_val, exc_tb)\u001b[0m\n\u001b[1;32m     70\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     71\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0misinstance\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mexc_val\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mBaseException\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 72\u001b[0;31m             \u001b[0;32mraise\u001b[0m \u001b[0mexc_val\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     73\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     74\u001b[0m         \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0madd\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mgraph\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mstack\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m-\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mgraph\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m<ipython-input-4-c0aea694bac2>\u001b[0m in \u001b[0;36m<cell line: 0>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      4\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      5\u001b[0m     \u001b[0;31m# try to access a layer of model that doesn't exist\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 6\u001b[0;31m     \u001b[0mmodel\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mtransformer\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mh\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m12\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0moutput\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m]\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      7\u001b[0m     \u001b[0moutput\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mmodel\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mlm_head\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0moutput\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0msave\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      8\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.11/dist-packages/nnsight/intervention/envoy.py\u001b[0m in \u001b[0;36m__getitem__\u001b[0;34m(self, key)\u001b[0m\n\u001b[1;32m    621\u001b[0m         \"\"\"\n\u001b[1;32m    622\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 623\u001b[0;31m         \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_children\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0mkey\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    624\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    625\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0m__len__\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;34m->\u001b[0m \u001b[0mint\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mIndexError\u001b[0m: list index out of range"
          ]
        }
      ],
      "source": [
        "prompt = 'The Eiffel Tower is in the city of'\n",
        "\n",
        "with model.trace(prompt) as tracer:\n",
        "\n",
        "    # try to access a layer of model that doesn't exist\n",
        "    model.transformer.h[12].output[0][:] = 0\n",
        "    output = model.lm_head.output.save()\n",
        "\n",
        "print(\"lm_head output = \",output)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "usZHg7l_YUYL"
      },
      "source": [
        "Great! Now we know that our list index was out of range within the tracing context, and if we expand to see the full message, we can tell that it's happening in line 7.\n",
        "\n",
        "Let's try again, now using the correct index for the final layer:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 224,
          "referenced_widgets": [
            "63dc1c0ce98d4539984bfc93114aa6c6",
            "1c6092db09c04e568b23ca71eb3f5f13",
            "54fc39e52ff94065a20594cc95a35942",
            "581c1d23c4424ce6b50aec079df134c4",
            "c7c13deb642f4af1bedd88c2afac85fd",
            "dea619854cbb42a2a700f7feef77d442",
            "c66f56814daa4745a0f14c75f85d0294",
            "5e25783cecd346f48959de00bc2cccec",
            "c700fe2789cb4f51a43ae1e29932d3fc",
            "e122506e7c7c454c952b89f306360da6",
            "53f9e552ea174246a10de4afc84272ac",
            "4fc680f767b04af881b52fd94ea07bba",
            "45f3295ab6e64cbda41f26c2d1581b08",
            "78018bc8cd974345b090ad38214e75c6",
            "56806395ddfe45f2b0245741a41e5bf5",
            "87e140a2c262488da5b393f37b51592c",
            "3b2050ce41594c69bf6efbacc76a574a",
            "15332cc2b6e84cfe937eece4ec884a72",
            "259617ed81dc4dd4b40f657ad63bec47",
            "16f914ebda5b44d28cd6aecc5ae742b7",
            "2130a5e5fd6d44ce8484c51d591e2d6e",
            "25af4170e72745a095f37ef4a481fab6"
          ]
        },
        "id": "Xu-WWnZsYpbw",
        "outputId": "7f33037a-066e-49c0-fef5-4fbb0267ac02"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "63dc1c0ce98d4539984bfc93114aa6c6",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "model.safetensors:   0%|          | 0.00/548M [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "4fc680f767b04af881b52fd94ea07bba",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "generation_config.json:   0%|          | 0.00/124 [00:00<?, ?B/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "lm_head output =  tensor([[[ -6.3267,  -6.1134,  -8.2121,  ..., -11.1459, -10.8880,  -6.1064],\n",
            "         [ -6.3267,  -6.1134,  -8.2121,  ..., -11.1459, -10.8880,  -6.1064],\n",
            "         [ -6.3267,  -6.1134,  -8.2121,  ..., -11.1459, -10.8880,  -6.1064],\n",
            "         ...,\n",
            "         [ -6.3267,  -6.1134,  -8.2121,  ..., -11.1459, -10.8880,  -6.1064],\n",
            "         [ -6.3267,  -6.1134,  -8.2121,  ..., -11.1459, -10.8880,  -6.1064],\n",
            "         [ -6.3267,  -6.1134,  -8.2121,  ..., -11.1459, -10.8880,  -6.1064]]],\n",
            "       device='cuda:0', grad_fn=<UnsafeViewBackward0>)\n"
          ]
        }
      ],
      "source": [
        "prompt = 'The Eiffel Tower is in the city of'\n",
        "\n",
        "with model.trace(prompt) as tracer:\n",
        "\n",
        "    # ablate last layer output\n",
        "    model.transformer.h[11].output[0][:] = 0\n",
        "    output = model.lm_head.output.save()\n",
        "\n",
        "print(\"lm_head output = \",output)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "yyzXpfTCn6Hu"
      },
      "source": [
        "The error messaging feature can be toggled using `nnsight.CONFIG.APP.DEBUG` which defaults to true."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "JTLOV5fKoIMS"
      },
      "outputs": [],
      "source": [
        "# Turn off debugging:\n",
        "import nnsight\n",
        "\n",
        "nnsight.CONFIG.APP.DEBUG = False\n",
        "nnsight.CONFIG.save()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Gl7KqYlUUKut"
      },
      "source": [
        "# .all()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0Zd4oyLAykM_"
      },
      "source": [
        "Sometimes you may want to recursively apply interventions to a model (e.g., when generating many tokens or for models like RNNs, where modules are called multiple times).\n",
        "\n",
        "\n",
        "\n",
        "*   Calling `.all()` on a model or its submodules will recursively apply its `.input` and `.output` across all iterations.\n",
        "*   When generating multiple tokens with `.generate` (see: [Multiple Token Generation](https://nnsight.net/notebooks/features/multiple_token/)), using `.all()` before applying an intervention will ensure that the model undergoes the intervention for *all* new tokens generated, not just the first.\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HTvh-jYt0ylW"
      },
      "source": [
        "## About"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0J6oxQfG0oE4"
      },
      "source": [
        "## .all() now streamlines multiple token generation"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "o8OenySKkk6m"
      },
      "source": [
        "With .all, applying interventions during multiple token generation becomes much easier. Let's test this out!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WWpVpxNk1DC5"
      },
      "source": [
        "We can use `.all()` to streamline the multiple token generation process. We simply call `.all` on the module where we are applying the intervention (in this case GPT-2's layers), apply our intervention, and append our hidden states (stored in an `nnsight.list()` object)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "oj9iFEWao1Od",
        "outputId": "ad07c368-ff8e-4b9b-b6df-aa47bc33e168"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Hidden state length:  3\n"
          ]
        }
      ],
      "source": [
        "# New: using .all():\n",
        "prompt = 'The Eiffel Tower is in the city of'\n",
        "layers = model.transformer.h\n",
        "n_new_tokens = 3\n",
        "with model.generate(prompt, max_new_tokens=n_new_tokens) as tracer:\n",
        "    hidden_states = nnsight.list().save() # Initialize & .save() nnsight list\n",
        "\n",
        "    # Call .all() to apply intervention to each new token\n",
        "    layers.all()\n",
        "\n",
        "    # Apply intervention - set first layer output to zero\n",
        "    layers[0].output[0][:] = 0\n",
        "\n",
        "    # Append desired hidden state post-intervention\n",
        "    hidden_states.append(layers[-1].output) # no need to call .save\n",
        "    # Don't need to loop or call .next()!\n",
        "\n",
        "print(\"Hidden state length: \",len(hidden_states))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pkT3YEyr4q5b"
      },
      "source": [
        "Easy! Note that because `.all()` is recursive, it will only work to append outputs called on children of the module that `.all()` was called on. See example below for more information. TL;DR: apply `.all()` on the highest-level accessed module if interventions and outputs have different hierarchies within model structure."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DbaaPivB4Xj_"
      },
      "source": [
        "### Note: (Old method) Applying interventions during multiple token generation without .all()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "U7LNx9aL1O5f"
      },
      "source": [
        "Without `.all()`, we would need to loop across each new generated token, saving the intervention for every generated token and calling `.next()` to move forward."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Ti0nvpuI5KfS",
        "outputId": "bdc00ff0-8980-41ad-ba2c-1bba317ae963"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Hidden state length:  3\n"
          ]
        }
      ],
      "source": [
        "# Old approach:\n",
        "prompt = 'The Eiffel Tower is in the city of'\n",
        "layers = model.transformer.h\n",
        "n_new_tokens = 3\n",
        "hidden_states = []\n",
        "with model.generate(prompt, max_new_tokens=n_new_tokens) as tracer:\n",
        "    for i in range(n_new_tokens):\n",
        "        # Apply intervention - set first layer output to zero\n",
        "        layers[0].output[0][:] = 0\n",
        "\n",
        "        # Append desired hidden state post-intervention\n",
        "        hidden_states.append(layers[-1].output.save())\n",
        "\n",
        "        # Move to next generated token\n",
        "        layers[0].next()\n",
        "\n",
        "print(\"Hidden state length: \",len(hidden_states))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "U5dmk3dj68dV"
      },
      "source": [
        "### Note: .all() recursive properties"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wGxmulNr0_WE"
      },
      "source": [
        "As mentioned, `.all()` is recursive and will work to append outputs called on children of the module that `.all` was called on. In this example, calling `.all()` on the model's layer modules will not recursively affect `model.lm_head.output` as it is not a child of layers."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "pLwXPnW-5unE",
        "outputId": "8af09e06-c563-4021-e47b-2e98f6c4ff98"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Hidden state length:  1\n"
          ]
        }
      ],
      "source": [
        "# A note on .all() recursive properties:\n",
        "prompt = 'The Eiffel Tower is in the city of'\n",
        "layers = model.transformer.h\n",
        "n_new_tokens = 3\n",
        "with model.generate(prompt, max_new_tokens=n_new_tokens) as tracer:\n",
        "    hidden_states = nnsight.list().save() # Initialize & .save() nnsight list\n",
        "\n",
        "    # Call .all() on layers\n",
        "    layers.all()\n",
        "\n",
        "    # Apply same intervention - set first layer output to zero\n",
        "    layers[0].output[0][:] = 0\n",
        "\n",
        "    # Append desired hidden state post-intervention\n",
        "    hidden_states.append(model.lm_head.output) # no need to call .save, it's already initialized\n",
        "\n",
        "print(\"Hidden state length: \",len(hidden_states)) # length is 1, meaning it only saved the first token generation"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IS2hNjEb6gd_"
      },
      "source": [
        "So, if you want to apply an intervention during multiple token generation while saving the state of a model component that isn't a child of that module, you can apply .`all()` to the full model."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "fNWHFuCg6lpC",
        "outputId": "ab8cb8b1-c056-4d3f-9fa7-4a37e613e868"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Hidden state length:  3\n"
          ]
        }
      ],
      "source": [
        "# Applying .all() to model fixes issue\n",
        "prompt = 'The Eiffel Tower is in the city of'\n",
        "layers = model.transformer.h\n",
        "n_new_tokens = 3\n",
        "with model.generate(prompt, max_new_tokens=n_new_tokens) as tracer:\n",
        "    hidden_states = nnsight.list().save() # Initialize & .save() nnsight list\n",
        "\n",
        "    # Call .all() on model\n",
        "    model.all()\n",
        "\n",
        "    # Apply same intervention - set first layer output to zero\n",
        "    layers[0].output[0][:] = 0\n",
        "\n",
        "    # Append desired hidden state post-intervention\n",
        "    hidden_states.append(model.lm_head.output) # no need to call .save\n",
        "\n",
        "print(\"Hidden state length: \",len(hidden_states)) # length is 3!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TrAvx6K0DscN"
      },
      "source": [
        "## Known Issues: `.all()`\n",
        "\n",
        "* IteratorEnvoy contexts can produce undesired behavior for subsequent operations defined <u>below</u> it that are not dependent on InterventionProxys.\n",
        "\n",
        "Example:\n",
        "```\n",
        "with lm.generate(\"Hello World!\", max_new_tokens=10):\n",
        "    hs_4 = nnsight.list().save()\n",
        "\n",
        "    with lm.transformer.h[4].all():\n",
        "        hs_4.append(lm.transformer.h[4].output)\n",
        "\n",
        "    hs_4.append(433)\n",
        "\n",
        "print(len(hs_4))\n",
        "```\n",
        "`>>> 20 # expected: 11`"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nuNMJsOdzrpF"
      },
      "source": [
        "# Syntax updates"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mJAp8_Pmj3kP"
      },
      "source": [
        "With `nnsight 0.4`, we now support `if` statements and `for` loops applied to proxies with traditional Python syntax! We also remove the need to call `.value` on a proxy output."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "o3JWZkgL-7Uj"
      },
      "source": [
        "## If Statements\n",
        "\n",
        "Previously, we would need to use `.cond()` to create a conditional context that would only execute upon meeting the logical conditions inside the `.cond()`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "cBPhM0vUj3JI",
        "outputId": "e1c6f677-5251-4b9c-809a-c47add938906"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Random Integer  tensor([5])  is Odd\n"
          ]
        }
      ],
      "source": [
        "import torch\n",
        "# Old method\n",
        "# model = LanguageModel('openai-community/gpt2', device_map='auto')\n",
        "\n",
        "with model.trace(\"The Eiffel Tower is in the city of\") as tracer:\n",
        "\n",
        "  rand_int = torch.randint(low=-10, high=10, size=(1,))\n",
        "\n",
        "  # To create the conditional context you need to put the\n",
        "  # condition within tracer.cond()\n",
        "  with tracer.cond(rand_int % 2 == 0):\n",
        "    tracer.log(\"Random Integer \", rand_int, \" is Even\")\n",
        "\n",
        "  with tracer.cond(rand_int % 2 == 1):\n",
        "    tracer.log(\"Random Integer \", rand_int, \" is Odd\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "B0rRtAYR_AIn"
      },
      "source": [
        "Now, we can use Python `if` statements within the tracing context to create a conditional context!\n",
        "\n",
        "*Note: Colab may be a little strangely with this feature the first time you run it - expect some lagging and warnings.*\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "UHBoLGZt--jC",
        "outputId": "5e4c7121-4db5-49d9-fd6a-541a27c5318c"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Random Integer  tensor([3])  is Odd\n"
          ]
        }
      ],
      "source": [
        "with model.trace(\"The Eiffel Tower is in the city of\") as tracer:\n",
        "\n",
        "  rand_int = torch.randint(low=-10, high=10, size=(1,))\n",
        "\n",
        "  # Since this if statement is inside the tracing context the if will\n",
        "  # create a conditional context and will only execute the intervention\n",
        "  # if this condition is met\n",
        "  if rand_int % 2 == 0:\n",
        "    tracer.log(\"Random Integer \", rand_int, \" is Even\")\n",
        "\n",
        "  if rand_int % 2 == 1:\n",
        "    tracer.log(\"Random Integer \", rand_int, \" is Odd\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "reSSkXLe71og"
      },
      "source": [
        "Note: If the conditional statements are outside the tracing context, `if` operates as in base Python.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kupHJGa9v_o5"
      },
      "source": [
        "`elif` statements should also work as `if` statements within the tracing context:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "4k5ubt7Fv-wY",
        "outputId": "074349f1-6201-49cf-c73c-96c9cb751d21"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Random Integer  tensor([-9])  is Odd\n"
          ]
        }
      ],
      "source": [
        "with model.trace(\"The Eiffel Tower is in the city of\") as tracer:\n",
        "\n",
        "  rand_int = torch.randint(low=-10, high=10, size=(1,))\n",
        "\n",
        "  # Since this if statement is inside the tracing context the if will\n",
        "  # create a conditional context and will only execute the intervention\n",
        "  # if this condition is met\n",
        "  if rand_int % 2 == 0:\n",
        "    tracer.log(\"Random Integer \", rand_int, \" is Even\")\n",
        "  elif rand_int % 2 == 1:\n",
        "    tracer.log(\"Random Integer \", rand_int, \" is Odd\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "A8dxj5cYRHjh"
      },
      "source": [
        "## For Loops"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xC5xw_rb3YXJ"
      },
      "source": [
        "With `nnsight 0.4`, you can now use `for` loops within a tracer context at scale. Previously, a `for` loop within a tracer context inside it resulted in creating intervention graphs over and over for each iteration - this is not scalable!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9gui9S8J-HZK"
      },
      "source": [
        "The `session.iter` context allows for scalable looping within sessions, but doesn't utilize traditional Python syntax:"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "VHglWqsJSRzv",
        "outputId": "f5adc897-b2c9-4fcc-ec48-042976a124f1"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "List:  [0, 1, 2]\n"
          ]
        }
      ],
      "source": [
        "# Old Method\n",
        "with model.session() as session:\n",
        "\n",
        "  li = nnsight.list() # an NNsight built-in list object\n",
        "  [li.append([num]) for num in range(0, 3)] # adding [0], [1], [2] to the list\n",
        "  li2 = nnsight.list().save()\n",
        "\n",
        "  # You can create nested Iterator contexts\n",
        "  with session.iter(li) as item:\n",
        "    with session.iter(item) as item_2:\n",
        "      li2.append(item_2)\n",
        "\n",
        "print(\"\\nList: \", li2)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SOTVCf6lBS9R"
      },
      "source": [
        "Now, you can use simple `for` loops within a tracer context to run an intervention loop at scale.\n",
        "\n",
        "*NOTE: inline for loops (i.e., `[x for x in <Proxy object>]`) are not currently supported.*"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Jnpp5VhdTO--",
        "outputId": "b7806e3c-0065-4307-a22e-4d93b9003738"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "List:  [0, 1, 2]\n"
          ]
        }
      ],
      "source": [
        "# New: Using Python for loops for iterative interventions\n",
        "with model.session() as session:\n",
        "\n",
        "    li = nnsight.list()\n",
        "    [li.append([num]) for num in range(0, 3)]\n",
        "    li2 = nnsight.list().save()\n",
        "\n",
        "    # Using regular for loops\n",
        "    for item in li:\n",
        "        for item_2 in item: # for loops can be nested!\n",
        "            li2.append(item_2)\n",
        "\n",
        "print(\"\\nList: \", li2)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "EpGXEcHM0Bs5"
      },
      "source": [
        "## `.value` injected into saved results\n",
        "\n",
        "Previously, directly using non-traceable functions (i.e., tokenizers) on a proxy returned from a tracing context required calling `.value` to access the proxy's numerical value. Calling traceable functions (like `print()` or `.argmax()`) on such proxies automatically returned the `.value`, making it optional to call `.value` in certain cases.\n",
        "\n",
        "```\n",
        "input = \"The Eiffel Tower is in the city of\"\n",
        "with model.trace(input):\n",
        "\n",
        "    l2_input = model.transformer.h[2].input.save()\n",
        "\n",
        "print(l2_input.value) # could optionally call .value\n",
        "print(l2_input) # but not required for traceable functions\n",
        "```"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6Sn1H93e-8Pd"
      },
      "source": [
        "Now with `nnsight 0.4`, the proxy's value is automatically injected into the variable name, negating any needs to call `.value` on proxies. Proxy variables will automatically be populated with their value upon exiting the tracing context. This is a breaking change, and calling `.value` on a proxy will now throw an error."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 323
        },
        "id": "_bYVfDL80Fn5",
        "outputId": "2236bd80-2076-4377-cc7d-63aa4b7a88fb"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "tensor([[[ 0.0386, -1.1676,  1.1246,  ..., -1.4047, -0.5742, -0.0668],\n",
            "         [ 0.1477, -0.4208,  1.3827,  ..., -1.6436, -0.1738, -1.1567],\n",
            "         [-0.1181, -0.5914, -0.9923,  ..., -0.8742, -0.1361,  1.0608],\n",
            "         ...,\n",
            "         [-1.3966,  0.8859,  0.1767,  ...,  0.0661,  0.6106, -0.6092],\n",
            "         [-2.9199,  0.3945, -3.4569,  ...,  1.2592,  0.2188, -0.5957],\n",
            "         [ 1.8395,  0.9940,  0.6617,  ..., -0.2802,  0.4978,  0.2308]]],\n",
            "       device='cuda:0', grad_fn=<AddBackward0>)\n"
          ]
        },
        {
          "ename": "AttributeError",
          "evalue": "'Tensor' object has no attribute 'value'",
          "output_type": "error",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mAttributeError\u001b[0m                            Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-15-02b94105ae06>\u001b[0m in \u001b[0;36m<cell line: 0>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      5\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      6\u001b[0m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ml2_input\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;31m# no need to call .value\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 7\u001b[0;31m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0ml2_input\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mvalue\u001b[0m\u001b[0;34m)\u001b[0m \u001b[0;31m# will throw an error\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
            "\u001b[0;31mAttributeError\u001b[0m: 'Tensor' object has no attribute 'value'"
          ]
        }
      ],
      "source": [
        "input = \"The Eiffel Tower is in the city of\"\n",
        "with model.trace(input):\n",
        "\n",
        "    l2_input = model.transformer.h[2].input.save()\n",
        "\n",
        "print(l2_input) # no need to call .value\n",
        "print(l2_input.value) # will throw an error"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "D9vGO0r968EC"
      },
      "source": [
        "## Turning off syntactic changes"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KBzJBKN06qB7"
      },
      "source": [
        "If you would like to turn off either the `if`/`for` functionality or the `.value` syntactic changes, you can apply the following changes to `nnsight.CONFIG`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "oM0ZFaRa65KX"
      },
      "outputs": [],
      "source": [
        "# Turn off if/for statements within tracing context:\n",
        "import nnsight\n",
        "\n",
        "nnsight.CONFIG.APP.CONTROL_FLOW_HANDLING = False\n",
        "nnsight.CONFIG.save()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "karGT1qv6qYr"
      },
      "outputs": [],
      "source": [
        "# Turn off .value injection:\n",
        "import nnsight\n",
        "\n",
        "nnsight.CONFIG.APP.FRAME_INJECTION = False\n",
        "nnsight.CONFIG.save()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "oyuEIDVJDDOi"
      },
      "source": [
        "## Known Issues: Syntax Update\n",
        "* Colab behaves a little strangely with these features the first time you run it - expect some lagging and warnings.\n",
        "\n",
        "* Inline Control Flow (for loops) are not supported.\n",
        "\n",
        "Example:\n",
        "```\n",
        "with lm.trace(\"Hello World!\"):\n",
        "    foo = nnsight.list([0, 1, 2, 3]).save()\n",
        "    [nnsight.log(item) for item in foo]\n",
        "\n",
        "```\n",
        "`>>> Error`\n",
        "\n",
        "* Value Injection is not supported for proxies referenced within objects.\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RNliY7nF1uPT"
      },
      "source": [
        "# vLLM Integration"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5tBDA0US7HAg"
      },
      "source": [
        "Our new update includes support for vLLM models using `nnsight`. [vLLM](https://github.com/vllm-project/vllm) is a popular library used for fast inference. By leveraging PagedAttention, dynamic batching, and Hugging Face model integration, vLLM makes inference more efficient and scalable for real-world applications."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xFGg4be_eJVx"
      },
      "source": [
        "## Setup"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "OVEQeIdhxOkG"
      },
      "source": [
        "You will need to install `nnsight 0.4`, `vllm`, and `triton 3.1.0` to use vLLM with NNsight."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "metadata": {
        "collapsed": true,
        "id": "3MG0_tXp_JI5"
      },
      "outputs": [],
      "source": [
        "from IPython.display import clear_output\n",
        "# install vllm\n",
        "!pip install vllm==0.6.4.post1\n",
        "\n",
        "# install triton 3.1.0\n",
        "!pip install triton==3.1.0\n",
        "\n",
        "clear_output()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lBFb58M0tZkN"
      },
      "source": [
        "**NOTE: you may need to restart your Colab session before the following step to properly load the `VLLM` model wrapper.**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cO0g7DRutzUp"
      },
      "source": [
        " Next, let's load in our NNsight-supported vLLM model. You can find vLLM-supported models [here](https://docs.vllm.ai/en/latest/models/supported_models.html). For this exercise, we will use GPT-2."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "9FHZysgYeTIU"
      },
      "outputs": [],
      "source": [
        "from nnsight.modeling.vllm import VLLM\n",
        "\n",
        "# NNsight's VLLM wrapper currently supports \"device = cuda\" and device = \"auto\"\n",
        "vllm = VLLM(\"gpt2\", device = \"auto\", dispatch = True) # See supported models: https://docs.vllm.ai/en/latest/models/supported_models.html\n",
        "print(vllm)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hrYn5-d8uJZb"
      },
      "source": [
        "## Interventions on vLLM models\n",
        "We now have a vLLM model that runs with `nnsight`. Let's try applying some interventions on it.\n",
        "\n",
        "Note that vLLM takes in sampling parameters including `temperature` and `top_p`. These parameters can be included in the `.trace()` or `.invoke()` contexts. For default model behavior, set `temperature = 0` and `top_p = 1`. For more information about parameters, reference the [vLLM documentation](https://docs.vllm.ai/en/latest/dev/sampling_params.html)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "RjSVuKBVue-m",
        "outputId": "b96e5d67-9fb8-4330-9f3b-d85f7a23ab45"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Processed prompts: 100%|██████████| 2/2 [00:00<00:00, 38.35it/s, est. speed input: 423.14 toks/s, output: 38.46 toks/s]\n"
          ]
        }
      ],
      "source": [
        "with vllm.trace(temperature=0.0, top_p=1.0, max_tokens=1) as tracer:\n",
        "  with tracer.invoke(\"The Eiffel Tower is located in the city of\"):\n",
        "    clean_logits = vllm.logits.output.save()\n",
        "\n",
        "  with tracer.invoke(\"The Eiffel Tower is located in the city of\"):\n",
        "    vllm.transformer.h[-2].mlp.output[:] = 0\n",
        "    corrupted_logits = vllm.logits.output.save()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "xMORJaB2uj7p",
        "outputId": "744f0c11-7529-4969-9cf1-b5c55da25dd1"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "CLEAN - The Eiffel Tower is located in the city of  Paris\n",
            "CORRUPTED - The Eiffel Tower is located in the city of  London\n"
          ]
        }
      ],
      "source": [
        "print(\"CLEAN - The Eiffel Tower is located in the city of\", vllm.tokenizer.decode(clean_logits.argmax(dim=-1)))\n",
        "print(\"CORRUPTED - The Eiffel Tower is located in the city of\", vllm.tokenizer.decode(corrupted_logits.argmax(dim=-1)))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9rLvuOlAv6zc"
      },
      "source": [
        "We've successfully performed an intervention on our vLLM model!"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "XpHPuAKTFRb3"
      },
      "source": [
        "## Sampled Token Traceability\n",
        "vLLM provides functionality to configure how each sequence samples its next token. Here's an example of how you can trace token sampling operations with the nnsight VLLM wrapper."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "xnSkk-GeFuNf",
        "outputId": "0b38ca8f-a978-4c3e-beea-f7ec4033b0fe"
      },
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Processed prompts: 100%|██████████| 1/1 [00:00<00:00,  5.84it/s, est. speed input: 53.99 toks/s, output: 17.99 toks/s]\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Samples:  [tensor([10346]), tensor([13]), tensor([6363])]\n",
            "Logits:  [tensor([[-109.0625, -107.9375, -111.6875,  ..., -115.3750, -116.5625,\n",
            "         -108.8750]], device='cuda:0', dtype=torch.float16), tensor([[-80.3125, -82.2500, -85.3750,  ..., -93.3750, -90.7500, -83.4375]],\n",
            "       device='cuda:0', dtype=torch.float16), tensor([[-110.0000, -109.3125, -110.4375,  ..., -120.9375, -119.3750,\n",
            "         -101.3750]], device='cuda:0', dtype=torch.float16)]\n"
          ]
        }
      ],
      "source": [
        "import nnsight\n",
        "with vllm.trace(\"Madison Square Garden is located in the city of\", temperature=0.8, top_p=0.95, max_tokens=3) as tracer:\n",
        "    samples = nnsight.list().save()\n",
        "    logits = nnsight.list().save()\n",
        "\n",
        "    for ii in range(3):\n",
        "        samples.append(vllm.samples.output)\n",
        "        vllm.samples.next()\n",
        "        logits.append(vllm.logits.output)\n",
        "        vllm.logits.next()\n",
        "\n",
        "print(\"Samples: \", samples)\n",
        "print(\"Logits: \", logits) # different than samples with current sampling parameters"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aL5y8M16wHsL"
      },
      "source": [
        "## Note: gradients are not supported with vLLM\n",
        "vLLM speeds up inference through its paged attention mechanism. This means that accessing gradients and backward passes are not supported for vLLM models. As such, calling gradient operations when using `nnsight` vLLM wrappers will throw an error."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aiNgpkdGDdBO"
      },
      "source": [
        "## Known Issues: vLLM Integration\n",
        "* The vllm.LLM engine performs max_tokens + 1 forward passes which can lead to undesired behavior if you are running interventions on all iterations of multi-token generation.\n",
        "\n",
        "Example:\n",
        "```\n",
        "with vllm_gpt2(\"Hello World!\", max_tokens=10):\n",
        "    logits = nnsight.list().save()\n",
        "    with vllm_gpt2.logits.all():\n",
        "        logits.append(vllm_gpt2.logits.output)\n",
        "\n",
        "print(len(logits))\n",
        "\n",
        "```\n",
        "`>>> 11 # expected: 10`"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xtSCr95WUN_s"
      },
      "source": [
        "# Streaming"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eV1H_LzRU0eo"
      },
      "source": [
        "Streaming enables users apply functions and datasets locally during remote model execution. This allows users to stream results for immediate consumption (i.e., seeing tokens as they are generated) or applying non-whitelisted functions such as model tokenizers, large local datasets, and more!\n",
        "\n",
        "*   `nnsight.local()` context sends values immediately to user's local machine from server\n",
        "*   Intervention graph is executed locally on downstream nodes\n",
        "*   Exiting local context uploads data back to server\n",
        "*   `@nnsight.trace` function decorator enables custom functions to be added to intervention graph when using `nnsight.local()`\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SLMF9D6KXVin"
      },
      "source": [
        "## `nnsight.local()`"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LhCNjtGl-oec"
      },
      "source": [
        "You may sometimes want to locally access and manipulate values during remote execution. Using `.local()` on a proxy, you can send remote content to your local machine and apply local functions. The intervention graph is then executed locally on downstream nodes until you exit the local context.\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "e3ZRmJi__uXE"
      },
      "source": [
        "There are a few use cases for streaming with `.local()`, including live chat generation and applying large datasets or non-whitelisted local functions to the intervention graph.\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "C7z4qaneMH7L"
      },
      "source": [
        "Now let's explore how streaming works. We'll start by grabbing some hidden states of the model and printing their value using `tracer.log()`. Without calling `nnsight.local()`, these operations will all occur remotely."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "PRSITdCKRBlq"
      },
      "outputs": [],
      "source": [
        "from nnsight import LanguageModel\n",
        "\n",
        "llama = LanguageModel(\"meta-llama/Meta-Llama-3.1-8B\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "hRA8MkG5MH7L"
      },
      "outputs": [],
      "source": [
        "# This will give you a remote LOG response because it's coming from the remote server\n",
        "with llama.trace(\"hello\", remote=True) as tracer:\n",
        "\n",
        "    hs = llama.model.layers[-1].output[0]\n",
        "\n",
        "    tracer.log(hs[0,0,0])\n",
        "\n",
        "    out =  llama.lm_head.output.save()\n",
        "\n",
        "print(out)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "06nlVWNmMH7M"
      },
      "source": [
        "Now, let's try the same operation using the `nnsight.local()` context. This will send the operations to get and print the hidden states to your local machine, changing how the logging message is formatted (local formatting instead of remote)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "H958j5eEMH7M"
      },
      "outputs": [],
      "source": [
        "# This will print locally because it's already local\n",
        "with llama.trace(\"hello\", remote=True) as tracer:\n",
        "\n",
        "    with nnsight.local():\n",
        "        hs = llama.model.layers[-1].output[0]\n",
        "        tracer.log(hs[0,0,0])\n",
        "\n",
        "    out =  llama.lm_head.output.save()\n",
        "\n",
        "print(out)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qA347WLnyexi"
      },
      "source": [
        "## `@nnsight.trace` function decorator"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "bTWtoGZzykkV"
      },
      "source": [
        "We can also use function decorators to create custom functions to be used during `.local` calls. This is a handy way to enable live streaming of a chat or to train probing classifiers on model hidden states."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2PhO5owrym1J"
      },
      "source": [
        "Let's try out `@nnsight.trace` and `nnsight.local()` to access a custom function during remote execution."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "DsIm3De-MH7M"
      },
      "outputs": [],
      "source": [
        "# first, let's define our function\n",
        "@nnsight.trace # decorator that enables this function to be added to the intervention graph\n",
        "def my_local_fn(value):\n",
        "    return value * 0\n",
        "\n",
        "# We use a local function to ablate some hidden states\n",
        "# This downloads the data for the .local context, and then uploads it back to set the value.\n",
        "with llama.generate(\"hello\", remote=True) as tracer:\n",
        "\n",
        "    hs = llama.model.layers[-1].output[0]\n",
        "\n",
        "    with nnsight.local():\n",
        "\n",
        "        hs = my_local_fn(hs)\n",
        "\n",
        "    llama.model.layers[-1].output[0][:] = hs\n",
        "\n",
        "    out =  llama.lm_head.output.save()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vYC1HZxOMH7M"
      },
      "source": [
        "Note that without calling `.local`, the remote API does not know about `my_local_fn` and will throw a whitelist error. A whitelist error occurs because you are being allowed access to the function."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "W7vNbMLrMH7M"
      },
      "outputs": [],
      "source": [
        "with llama.trace(\"hello\", remote=True) as tracer:\n",
        "\n",
        "    hs = llama.model.layers[-1].output[0]\n",
        "\n",
        "    hs = my_local_fn(hs) # no .local - will cause an error\n",
        "\n",
        "    llama.model.layers[-1].output[0][:] = hs * 2\n",
        "\n",
        "    out =  llama.lm_head.output.save()\n",
        "\n",
        "print(out)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "M-aZ3YzOAiNr"
      },
      "source": [
        "## Example: Live-streaming remote chat\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "I90m5IuuMH7M"
      },
      "source": [
        "Now that we can access data within the tracing context on our local computer, we can apply non-whitelisted functions, such as the model's tokenizer, within our tracing context.\n",
        "\n",
        "Let's build a decoding function that will decode tokens into words and print the result."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "0oub4XlnMH7M"
      },
      "outputs": [],
      "source": [
        "@nnsight.trace\n",
        "def my_decoding_function(tokens, model, max_length=80, state=None):\n",
        "    # Initialize state if not provided\n",
        "    if state is None:\n",
        "        state = {'current_line': '', 'current_line_length': 0}\n",
        "\n",
        "    token = tokens[-1] # only use last token\n",
        "\n",
        "    # Decode the token\n",
        "    decoded_token = llama.tokenizer.decode(token).encode(\"unicode_escape\").decode()\n",
        "\n",
        "    if decoded_token == '\\\\n':  # Handle explicit newline tokens\n",
        "        # Print the current line and reset state\n",
        "        print('',flush=True)\n",
        "        state['current_line'] = ''\n",
        "        state['current_line_length'] = 0\n",
        "    else:\n",
        "        # Check if adding the token would exceed the max length\n",
        "        if state['current_line_length'] + len(decoded_token) > max_length:\n",
        "            print('',flush=True)\n",
        "            state['current_line'] = decoded_token  # Start a new line with the current token\n",
        "            state['current_line_length'] = len(decoded_token)\n",
        "            print(state['current_line'], flush=True, end=\"\")  # Print the current line\n",
        "        else:\n",
        "            # Add a space if the line isn't empty and append the token\n",
        "            if state['current_line']:\n",
        "                state['current_line'] += decoded_token\n",
        "            else:\n",
        "                state['current_line'] = decoded_token\n",
        "            state['current_line_length'] += len(decoded_token)\n",
        "            print(state['current_line'], flush=True, end=\"\")  # Print the current line\n",
        "\n",
        "    return state"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2qbXSwMGOhd-"
      },
      "source": [
        "Now we can decode and print our model outputs throughout token generation by accessing our decoding function through `nnsight.local()`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "jzjWIFbgMH7M"
      },
      "outputs": [],
      "source": [
        "import torch\n",
        "\n",
        "nnsight.CONFIG.APP.REMOTE_LOGGING = False\n",
        "\n",
        "prompt = \"A press release is an official statement delivered to members of the news media for the purpose of\"\n",
        "# prompt = \"Your favorite board game is\"\n",
        "\n",
        "print(\"Prompt: \",prompt,'\\n', end =\"\")\n",
        "\n",
        "# Initialize the state for decoding\n",
        "state = {'current_line': '', 'current_line_length': 0}\n",
        "\n",
        "with llama.generate(prompt, remote=True, max_new_tokens = 50) as generator:\n",
        "    # Call .all() to apply to each new token\n",
        "    llama.all()\n",
        "\n",
        "    all_tokens = nnsight.list().save()\n",
        "\n",
        "    # Access model output\n",
        "    out = llama.lm_head.output.save()\n",
        "\n",
        "    # Apply softmax to obtain probabilities and save the result\n",
        "    probs = torch.nn.functional.softmax(out, dim=-1)\n",
        "    max_probs = torch.max(probs, dim=-1)\n",
        "    tokens = max_probs.indices.cpu().tolist()\n",
        "    all_tokens.append(tokens[0]).save()\n",
        "\n",
        "    with nnsight.local():\n",
        "        state = my_decoding_function(tokens[0], llama, max_length=20, state=state)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "JY8p_7Zj81B4"
      },
      "source": [
        "# General Considerations\n",
        "* `Tracer.cond(…)` and `Tracer.iter(…)` are still supported.\n",
        "\n",
        "* vLLM <U>does not</u> come as a pre-installed dependency of `nnsight`.\n",
        "\n",
        "* `nnsight` supports `vllm==0.6.4.post1`\n",
        "\n",
        "* vLLM support only includes `cuda` and `auto` devices at the moment.\n",
        "\n",
        "* vLLM models <u>do not</u> support gradients.\n",
        "\n",
        "* The `@nnsight.trace` decorator does not enable user-defined operations to be executed remotely. Something coming soon for that..."
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "collapsed_sections": [
        "DbaaPivB4Xj_"
      ],
      "gpuType": "T4",
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.10.13"
    },
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "0325fe95b3e3432497fc68317b2126d4": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0650393001b34123a0e18663e6aa0fcb": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0e4b52697a87475882e40b3ca4a2e77d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_60627b23fe934d01b4a4af1f592fbd55",
            "max": 26,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_a806062fe502459a93ee212e57af2e66",
            "value": 26
          }
        },
        "0f3eaadfc2c34384afd2979417d94812": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_a2eafc8f9262412793f35b372eb300eb",
              "IPY_MODEL_cb3a1a3578554cac9cec37d9b0390c1a",
              "IPY_MODEL_17f30f75790e4273b3e475f433c4760d"
            ],
            "layout": "IPY_MODEL_1046b0563cb64ce68fc7b1b2f5d3d65d"
          }
        },
        "1046b0563cb64ce68fc7b1b2f5d3d65d": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "15332cc2b6e84cfe937eece4ec884a72": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "16f914ebda5b44d28cd6aecc5ae742b7": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "176749c3ecd24b829a78f892dee8af91": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_0325fe95b3e3432497fc68317b2126d4",
            "placeholder": "​",
            "style": "IPY_MODEL_fabbb16b5e7942f493b7dc92279715fd",
            "value": " 1.36M/1.36M [00:00&lt;00:00, 15.2MB/s]"
          }
        },
        "17f30f75790e4273b3e475f433c4760d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2e14c72eaf6644bd9d656513c302c278",
            "placeholder": "​",
            "style": "IPY_MODEL_30af0b2c974d418cb40b30403a2a90cf",
            "value": " 665/665 [00:00&lt;00:00, 39.8kB/s]"
          }
        },
        "1ac5508e75154bd18275d6f6775242b7": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a22aff0a7b33470b970a32269a9a0f08",
            "placeholder": "​",
            "style": "IPY_MODEL_be8e4e0c8e0c482897194f08c77f82a0",
            "value": " 456k/456k [00:00&lt;00:00, 2.80MB/s]"
          }
        },
        "1c6092db09c04e568b23ca71eb3f5f13": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_dea619854cbb42a2a700f7feef77d442",
            "placeholder": "​",
            "style": "IPY_MODEL_c66f56814daa4745a0f14c75f85d0294",
            "value": "model.safetensors: 100%"
          }
        },
        "2130a5e5fd6d44ce8484c51d591e2d6e": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "259617ed81dc4dd4b40f657ad63bec47": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "25af4170e72745a095f37ef4a481fab6": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "2e14c72eaf6644bd9d656513c302c278": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2f851281cd2448f8a36c9b538931e0c9": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_637e15c89e6847098b27f7a4f3482a63",
            "placeholder": "​",
            "style": "IPY_MODEL_e039919f976f4d5fbebb5117ec53d57e",
            "value": "tokenizer.json: 100%"
          }
        },
        "30af0b2c974d418cb40b30403a2a90cf": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "3160c45575ad4426b2abcfaccd27d41c": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "33c1fe5504d1441e805a02fb4693acd7": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_8bf1249ac7aa46b094bc899d432153fa",
              "IPY_MODEL_c60da3ab273d4439a0baec6ae7c216e4",
              "IPY_MODEL_9ef4dc73855c4fa69656f75110ac390d"
            ],
            "layout": "IPY_MODEL_4b1d8724a6b247e0a3dafe2f6b9de178"
          }
        },
        "33cc1f2b3b884a65a4a942cde1662b8f": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3adee7a17b2a4f50b9ff36fe10a3b701": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "3b2050ce41594c69bf6efbacc76a574a": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3c1c03b756c1401da717b3b62842a1f9": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "416faee108c44dce9e1b3527e82acb9b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "45f3295ab6e64cbda41f26c2d1581b08": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_3b2050ce41594c69bf6efbacc76a574a",
            "placeholder": "​",
            "style": "IPY_MODEL_15332cc2b6e84cfe937eece4ec884a72",
            "value": "generation_config.json: 100%"
          }
        },
        "4b1d8724a6b247e0a3dafe2f6b9de178": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4d7f03cfa9944554b093b3e5c691db92": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4eb7af991ebe4bdd855d3cdd50184495": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4fc680f767b04af881b52fd94ea07bba": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_45f3295ab6e64cbda41f26c2d1581b08",
              "IPY_MODEL_78018bc8cd974345b090ad38214e75c6",
              "IPY_MODEL_56806395ddfe45f2b0245741a41e5bf5"
            ],
            "layout": "IPY_MODEL_87e140a2c262488da5b393f37b51592c"
          }
        },
        "53f9e552ea174246a10de4afc84272ac": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "54fc39e52ff94065a20594cc95a35942": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_5e25783cecd346f48959de00bc2cccec",
            "max": 548105171,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_c700fe2789cb4f51a43ae1e29932d3fc",
            "value": 548105171
          }
        },
        "56806395ddfe45f2b0245741a41e5bf5": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2130a5e5fd6d44ce8484c51d591e2d6e",
            "placeholder": "​",
            "style": "IPY_MODEL_25af4170e72745a095f37ef4a481fab6",
            "value": " 124/124 [00:00&lt;00:00, 7.15kB/s]"
          }
        },
        "581c1d23c4424ce6b50aec079df134c4": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e122506e7c7c454c952b89f306360da6",
            "placeholder": "​",
            "style": "IPY_MODEL_53f9e552ea174246a10de4afc84272ac",
            "value": " 548M/548M [00:02&lt;00:00, 220MB/s]"
          }
        },
        "59f8564845764c9e9891274216be83ff": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5c5d0fc4f29141fc95008070c0b077d3": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_4d7f03cfa9944554b093b3e5c691db92",
            "placeholder": "​",
            "style": "IPY_MODEL_732fe1ca67b74026805de4688d81e6e3",
            "value": "merges.txt: 100%"
          }
        },
        "5e25783cecd346f48959de00bc2cccec": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "60627b23fe934d01b4a4af1f592fbd55": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "637e15c89e6847098b27f7a4f3482a63": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6386746d591e4e4cb83a18dcc4481678": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "63dc1c0ce98d4539984bfc93114aa6c6": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_1c6092db09c04e568b23ca71eb3f5f13",
              "IPY_MODEL_54fc39e52ff94065a20594cc95a35942",
              "IPY_MODEL_581c1d23c4424ce6b50aec079df134c4"
            ],
            "layout": "IPY_MODEL_c7c13deb642f4af1bedd88c2afac85fd"
          }
        },
        "6909cf23963d4347820b3717b62b1cfe": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6fc349dcab8d43309faaa08775f445ff": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "732fe1ca67b74026805de4688d81e6e3": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "78018bc8cd974345b090ad38214e75c6": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_259617ed81dc4dd4b40f657ad63bec47",
            "max": 124,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_16f914ebda5b44d28cd6aecc5ae742b7",
            "value": 124
          }
        },
        "7c3d6693887b4c29b3d63afd37e4292e": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "87e140a2c262488da5b393f37b51592c": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "886f12cb69c2435d978eb2d33534763a": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "8bf1249ac7aa46b094bc899d432153fa": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6909cf23963d4347820b3717b62b1cfe",
            "placeholder": "​",
            "style": "IPY_MODEL_f6a764157eab4ee7b24ce3dc3e317222",
            "value": "vocab.json: 100%"
          }
        },
        "9a2747e522c94c00af3f9d0bbd54cc1d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_7c3d6693887b4c29b3d63afd37e4292e",
            "max": 1355256,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_6386746d591e4e4cb83a18dcc4481678",
            "value": 1355256
          }
        },
        "9b161849be8a4464a486e6270f3c0f3b": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9ef4dc73855c4fa69656f75110ac390d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e60f4ecadf75463ca5fc5f934954e144",
            "placeholder": "​",
            "style": "IPY_MODEL_d5f42dda1612435f8eb053fe29f711c0",
            "value": " 1.04M/1.04M [00:00&lt;00:00, 3.50MB/s]"
          }
        },
        "a1595afd22fa4d999a651e1ff63f4279": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_5c5d0fc4f29141fc95008070c0b077d3",
              "IPY_MODEL_c410466db4ac4d878c3aac7fe4a0df5b",
              "IPY_MODEL_1ac5508e75154bd18275d6f6775242b7"
            ],
            "layout": "IPY_MODEL_6fc349dcab8d43309faaa08775f445ff"
          }
        },
        "a22aff0a7b33470b970a32269a9a0f08": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a2eafc8f9262412793f35b372eb300eb": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_886f12cb69c2435d978eb2d33534763a",
            "placeholder": "​",
            "style": "IPY_MODEL_cb0ab1cce5364b44ba85e65ccf44d0c9",
            "value": "config.json: 100%"
          }
        },
        "a806062fe502459a93ee212e57af2e66": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "be8e4e0c8e0c482897194f08c77f82a0": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "c3b4c47b73d544eaba67d5828f1f742d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_c924f41345d34f72a00ed3b21f84de82",
              "IPY_MODEL_0e4b52697a87475882e40b3ca4a2e77d",
              "IPY_MODEL_c4c1c2ccc4014520bb265d21550fb133"
            ],
            "layout": "IPY_MODEL_3160c45575ad4426b2abcfaccd27d41c"
          }
        },
        "c410466db4ac4d878c3aac7fe4a0df5b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_3c1c03b756c1401da717b3b62842a1f9",
            "max": 456318,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_416faee108c44dce9e1b3527e82acb9b",
            "value": 456318
          }
        },
        "c4c1c2ccc4014520bb265d21550fb133": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_9b161849be8a4464a486e6270f3c0f3b",
            "placeholder": "​",
            "style": "IPY_MODEL_f2ca144c6f8d4537ac8e2e29c8109454",
            "value": " 26.0/26.0 [00:00&lt;00:00, 1.49kB/s]"
          }
        },
        "c60da3ab273d4439a0baec6ae7c216e4": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_59f8564845764c9e9891274216be83ff",
            "max": 1042301,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_3adee7a17b2a4f50b9ff36fe10a3b701",
            "value": 1042301
          }
        },
        "c66f56814daa4745a0f14c75f85d0294": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "c700fe2789cb4f51a43ae1e29932d3fc": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "c7c13deb642f4af1bedd88c2afac85fd": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "c924f41345d34f72a00ed3b21f84de82": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_4eb7af991ebe4bdd855d3cdd50184495",
            "placeholder": "​",
            "style": "IPY_MODEL_c96d870993144ed09e54de5dc00fad6d",
            "value": "tokenizer_config.json: 100%"
          }
        },
        "c96d870993144ed09e54de5dc00fad6d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "cb0ab1cce5364b44ba85e65ccf44d0c9": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "cb3a1a3578554cac9cec37d9b0390c1a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_0650393001b34123a0e18663e6aa0fcb",
            "max": 665,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_cdfa2b53fdd14d24b9ccc0920f34bd69",
            "value": 665
          }
        },
        "cdfa2b53fdd14d24b9ccc0920f34bd69": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "d5f42dda1612435f8eb053fe29f711c0": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "d86efefd42444836b476a5caecccdb75": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_2f851281cd2448f8a36c9b538931e0c9",
              "IPY_MODEL_9a2747e522c94c00af3f9d0bbd54cc1d",
              "IPY_MODEL_176749c3ecd24b829a78f892dee8af91"
            ],
            "layout": "IPY_MODEL_33cc1f2b3b884a65a4a942cde1662b8f"
          }
        },
        "dea619854cbb42a2a700f7feef77d442": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e039919f976f4d5fbebb5117ec53d57e": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "e122506e7c7c454c952b89f306360da6": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e60f4ecadf75463ca5fc5f934954e144": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f2ca144c6f8d4537ac8e2e29c8109454": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "f6a764157eab4ee7b24ce3dc3e317222": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "fabbb16b5e7942f493b7dc92279715fd": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        }
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}