3_Text_Generation_with_RNNs.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "gpuType": "L4"
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "# Text Generation with RNNs\n",
        "\n",
        "In this notebook, we will build a simple Recurrent Neural Network (RNN) using Keras to generate text sequences in the style of Shakespeare. We will use a dataset of Shakespeare's works, preprocess the text, train the RNN, and generate new text based on the learned patterns."
      ],
      "metadata": {
        "id": "9eSnzkZHZecM"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "rNwJlaazZcWA"
      },
      "outputs": [],
      "source": [
        "import numpy as np\n",
        "import random\n",
        "import tensorflow as tf\n",
        "from tensorflow.keras.models import Sequential\n",
        "from tensorflow.keras.layers import Dense, Embedding, SimpleRNN\n",
        "from tensorflow.keras.optimizers import RMSprop\n",
        "from tensorflow.keras.preprocessing.text import Tokenizer\n",
        "from tensorflow.keras.preprocessing.sequence import pad_sequences"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Download the dataset\n",
        "!wget -q -O shakespeare.txt https://www.gutenberg.org/cache/epub/100/pg100.txt\n",
        "\n",
        "# Load the dataset\n",
        "with open('shakespeare.txt', 'r', encoding='utf-8') as file:\n",
        "    text = file.read()\n",
        "\n",
        "# Display the first 1000 characters\n",
        "print(text[:1000])"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Giwv32g0ZpTP",
        "outputId": "012702a5-654c-48cf-886b-bff7c3a176cc"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "﻿The Project Gutenberg eBook of The Complete Works of William Shakespeare\n",
            "    \n",
            "This ebook is for the use of anyone anywhere in the United States and\n",
            "most other parts of the world at no cost and with almost no restrictions\n",
            "whatsoever. You may copy it, give it away or re-use it under the terms\n",
            "of the Project Gutenberg License included with this ebook or online\n",
            "at www.gutenberg.org. If you are not located in the United States,\n",
            "you will have to check the laws of the country where you are located\n",
            "before using this eBook.\n",
            "\n",
            "Title: The Complete Works of William Shakespeare\n",
            "\n",
            "Author: William Shakespeare\n",
            "\n",
            "Release date: January 1, 1994 [eBook #100]\n",
            "                Most recently updated: January 18, 2024\n",
            "\n",
            "Language: English\n",
            "\n",
            "\n",
            "\n",
            "*** START OF THE PROJECT GUTENBERG EBOOK THE COMPLETE WORKS OF WILLIAM SHAKESPEARE ***\n",
            "﻿The Complete Works of William Shakespeare\n",
            "\n",
            "by William Shakespeare\n",
            "\n",
            "\n",
            "\n",
            "\n",
            "                    Contents\n",
            "\n",
            "    THE SONNETS\n",
            "    ALL’S WELL THAT ENDS WELL\n",
            "    THE TRAGEDY OF ANTONY AND CLEOPATRA\n",
            "   \n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "We preprocess the text data for training a character-level RNN. We begin by utilizing Keras's Tokenizer with `char_level=True` to tokenize the text at the character level, assigning a unique integer index to each character. This approach allows the model to learn patterns at the granularity of individual characters.\n",
        "\n",
        "The text is then converted into a sequence of integers, representing each character by its corresponding index. We define sequences of 40 characters as input, with the subsequent character serving as the target output. This setup allows the RNN to learn to predict the next character in a sequence, facilitating text generation.\n",
        "\n",
        "To prepare the target data for model training, we one-hot encode the output characters.\n",
        "\n",
        "Finally, the input `X` and output `y` data are converted into NumPy arrays, ensuring compatibility with deep learning models."
      ],
      "metadata": {
        "id": "FyGWVW-GQaZJ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Tokenization\n",
        "tokenizer = Tokenizer(char_level=True)  # Character-level tokenization\n",
        "tokenizer.fit_on_texts(text)\n",
        "total_chars = len(tokenizer.word_index)\n",
        "\n",
        "# Convert text to sequences of integers\n",
        "sequences = tokenizer.texts_to_sequences([text])[0]\n",
        "\n",
        "# Define sequence length\n",
        "sequence_length = 40\n",
        "\n",
        "# Create input-output pairs\n",
        "X = []\n",
        "y = []\n",
        "for i in range(0, len(sequences) - sequence_length):\n",
        "    X.append(sequences[i:i + sequence_length])\n",
        "    y.append(sequences[i + sequence_length])\n",
        "\n",
        "# Convert to NumPy arrays\n",
        "X = np.array(X)\n",
        "y = np.array(y)\n",
        "\n",
        "# One-hot encode the output variable\n",
        "y = tf.keras.utils.to_categorical(y, num_classes=total_chars + 1)\n",
        "\n",
        "print(f\"Number of sequences: {X.shape[0]}\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Nf4CT2w3ZsSs",
        "outputId": "eda2dda1-1b78-449a-8edb-0615e15e85be"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Number of sequences: 5378624\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "X will contain sequences like:\n",
        "```\n",
        "[h, e, l, l, o]\n",
        "[e, l, l, o, _]\n",
        "[l, l, o, _, w]\n",
        "```\n",
        "And so on, where each letter is replaced by its corresponding integer index.\n",
        "\n",
        " `y` is a one-hot encoded array where each row corresponds to the true next character following the sequence in `X`."
      ],
      "metadata": {
        "id": "yDx3FP2ISMAF"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def build_rnn_model(input_shape, num_classes):\n",
        "    \"\"\"\n",
        "    Build a simple RNN model using Keras Sequential API.\n",
        "\n",
        "    Parameters:\n",
        "    - input_shape: Shape of the input data (sequence_length)\n",
        "    - num_classes: Number of unique characters (output size)\n",
        "\n",
        "    Returns:\n",
        "    - model: Compiled RNN model\n",
        "    \"\"\"\n",
        "    model = Sequential()\n",
        "    model.add(Embedding(input_dim=num_classes, output_dim=32, input_length=input_shape))  # Converts character indices into dense vectors of fixed size\n",
        "    model.add(SimpleRNN(128, return_sequences=False))\n",
        "    model.add(Dense(num_classes, activation='softmax'))\n",
        "\n",
        "    model.compile(optimizer=RMSprop(learning_rate=0.01), loss='categorical_crossentropy', metrics=['accuracy'])\n",
        "\n",
        "    return model\n",
        "\n",
        "# Build the model\n",
        "input_shape = X.shape[1]  # sequence_length\n",
        "num_classes = total_chars + 1\n",
        "rnn_model = build_rnn_model(input_shape, num_classes)\n",
        "rnn_model.summary()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 269
        },
        "id": "-8c3ILpgZx3j",
        "outputId": "7e44edbb-46a1-4a24-e67f-26fd2c9a2b1a"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/keras/src/layers/core/embedding.py:90: UserWarning: Argument `input_length` is deprecated. Just remove it.\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "\u001b[1mModel: \"sequential\"\u001b[0m\n"
            ],
            "text/html": [
              "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\">Model: \"sequential\"</span>\n",
              "</pre>\n"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓\n",
              "┃\u001b[1m \u001b[0m\u001b[1mLayer (type)                        \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mOutput Shape               \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1m        Param #\u001b[0m\u001b[1m \u001b[0m┃\n",
              "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩\n",
              "│ embedding (\u001b[38;5;33mEmbedding\u001b[0m)                │ ?                           │     \u001b[38;5;34m0\u001b[0m (unbuilt) │\n",
              "├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤\n",
              "│ simple_rnn (\u001b[38;5;33mSimpleRNN\u001b[0m)               │ ?                           │     \u001b[38;5;34m0\u001b[0m (unbuilt) │\n",
              "├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤\n",
              "│ dense (\u001b[38;5;33mDense\u001b[0m)                        │ ?                           │     \u001b[38;5;34m0\u001b[0m (unbuilt) │\n",
              "└──────────────────────────────────────┴─────────────────────────────┴─────────────────┘\n"
            ],
            "text/html": [
              "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓\n",
              "┃<span style=\"font-weight: bold\"> Layer (type)                         </span>┃<span style=\"font-weight: bold\"> Output Shape                </span>┃<span style=\"font-weight: bold\">         Param # </span>┃\n",
              "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩\n",
              "│ embedding (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Embedding</span>)                │ ?                           │     <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> (unbuilt) │\n",
              "├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤\n",
              "│ simple_rnn (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">SimpleRNN</span>)               │ ?                           │     <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> (unbuilt) │\n",
              "├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤\n",
              "│ dense (<span style=\"color: #0087ff; text-decoration-color: #0087ff\">Dense</span>)                        │ ?                           │     <span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> (unbuilt) │\n",
              "└──────────────────────────────────────┴─────────────────────────────┴─────────────────┘\n",
              "</pre>\n"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "\u001b[1m Total params: \u001b[0m\u001b[38;5;34m0\u001b[0m (0.00 B)\n"
            ],
            "text/html": [
              "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Total params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> (0.00 B)\n",
              "</pre>\n"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "\u001b[1m Trainable params: \u001b[0m\u001b[38;5;34m0\u001b[0m (0.00 B)\n"
            ],
            "text/html": [
              "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Trainable params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> (0.00 B)\n",
              "</pre>\n"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "\u001b[1m Non-trainable params: \u001b[0m\u001b[38;5;34m0\u001b[0m (0.00 B)\n"
            ],
            "text/html": [
              "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\"><span style=\"font-weight: bold\"> Non-trainable params: </span><span style=\"color: #00af00; text-decoration-color: #00af00\">0</span> (0.00 B)\n",
              "</pre>\n"
            ]
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# Train the model\n",
        "history = rnn_model.fit(X, y, validation_split=0.2, epochs=20, batch_size=128, verbose=1)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Z3i1jeaLZ1hh",
        "outputId": "f57083d3-f3d5-442f-d891-57e729355ba0"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/20\n",
            "\u001b[1m33617/33617\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m229s\u001b[0m 7ms/step - accuracy: 0.3564 - loss: 2.2180 - val_accuracy: 0.3504 - val_loss: 2.1908\n",
            "Epoch 2/20\n",
            "\u001b[1m33617/33617\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m211s\u001b[0m 6ms/step - accuracy: 0.3872 - loss: 2.0872 - val_accuracy: 0.3712 - val_loss: 2.2234\n",
            "Epoch 3/20\n",
            "\u001b[1m33617/33617\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m214s\u001b[0m 6ms/step - accuracy: 0.3916 - loss: 2.0767 - val_accuracy: 0.3714 - val_loss: 2.2418\n",
            "Epoch 4/20\n",
            "\u001b[1m33617/33617\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m217s\u001b[0m 6ms/step - accuracy: 0.3958 - loss: 2.0652 - val_accuracy: 0.3589 - val_loss: 2.3231\n",
            "Epoch 5/20\n",
            "\u001b[1m33617/33617\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m206s\u001b[0m 6ms/step - accuracy: 0.3962 - loss: 2.0628 - val_accuracy: 0.3721 - val_loss: 2.2488\n",
            "Epoch 6/20\n",
            "\u001b[1m33617/33617\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m212s\u001b[0m 6ms/step - accuracy: 0.3983 - loss: 2.0592 - val_accuracy: 0.3868 - val_loss: 2.3711\n",
            "Epoch 7/20\n",
            "\u001b[1m33617/33617\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m206s\u001b[0m 6ms/step - accuracy: 0.4009 - loss: 2.0525 - val_accuracy: 0.4022 - val_loss: 2.3065\n",
            "Epoch 8/20\n",
            "\u001b[1m33617/33617\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m208s\u001b[0m 6ms/step - accuracy: 0.3999 - loss: 2.0582 - val_accuracy: 0.3554 - val_loss: 2.3380\n",
            "Epoch 9/20\n",
            "\u001b[1m33617/33617\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m213s\u001b[0m 6ms/step - accuracy: 0.4018 - loss: 2.0597 - val_accuracy: 0.3945 - val_loss: 2.2940\n",
            "Epoch 10/20\n",
            "\u001b[1m33617/33617\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m205s\u001b[0m 6ms/step - accuracy: 0.4030 - loss: 2.0635 - val_accuracy: 0.3993 - val_loss: 2.3735\n",
            "Epoch 11/20\n",
            "\u001b[1m33617/33617\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m205s\u001b[0m 6ms/step - accuracy: 0.4038 - loss: 2.0573 - val_accuracy: 0.3877 - val_loss: 2.3594\n",
            "Epoch 12/20\n",
            "\u001b[1m33617/33617\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m207s\u001b[0m 6ms/step - accuracy: 0.4046 - loss: 2.0621 - val_accuracy: 0.3899 - val_loss: 2.3324\n",
            "Epoch 13/20\n",
            "\u001b[1m33617/33617\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m206s\u001b[0m 6ms/step - accuracy: 0.4041 - loss: 2.0670 - val_accuracy: 0.3622 - val_loss: 2.3898\n",
            "Epoch 14/20\n",
            "\u001b[1m33617/33617\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m209s\u001b[0m 6ms/step - accuracy: 0.4053 - loss: 2.0674 - val_accuracy: 0.3908 - val_loss: 2.4496\n",
            "Epoch 15/20\n",
            "\u001b[1m33617/33617\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m208s\u001b[0m 6ms/step - accuracy: 0.4056 - loss: 2.0640 - val_accuracy: 0.3858 - val_loss: 2.4343\n",
            "Epoch 16/20\n",
            "\u001b[1m33617/33617\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m206s\u001b[0m 6ms/step - accuracy: 0.4057 - loss: 2.0777 - val_accuracy: 0.3819 - val_loss: 2.4603\n",
            "Epoch 17/20\n",
            "\u001b[1m33617/33617\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m207s\u001b[0m 6ms/step - accuracy: 0.4059 - loss: 2.0766 - val_accuracy: 0.3659 - val_loss: 2.6216\n",
            "Epoch 18/20\n",
            "\u001b[1m33617/33617\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m206s\u001b[0m 6ms/step - accuracy: 0.4083 - loss: 2.0748 - val_accuracy: 0.3741 - val_loss: 2.5386\n",
            "Epoch 19/20\n",
            "\u001b[1m33617/33617\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m206s\u001b[0m 6ms/step - accuracy: 0.4073 - loss: 2.0844 - val_accuracy: 0.3947 - val_loss: 2.5487\n",
            "Epoch 20/20\n",
            "\u001b[1m33617/33617\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m205s\u001b[0m 6ms/step - accuracy: 0.4074 - loss: 2.0917 - val_accuracy: 0.3791 - val_loss: 2.5645\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "The `generate_text` function uses the trained RNN to generate text starting from a given seed text. It tokenizes the last `sequence_length` characters of the seed to form an input sequence, which is then fed into the model to predict the next character. The predicted character, determined by the highest probability from the model's output, is appended to the growing text sequence. This process is repeated for a specified number of characters, building the generated text iteratively. The final result is a string that extends the seed text, reflecting patterns learned by the model during training."
      ],
      "metadata": {
        "id": "sXgsWt_bTiIw"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def generate_text(model, tokenizer, seed_text, length=200):\n",
        "    \"\"\"\n",
        "    Generate text using a trained model and a seed text.\n",
        "\n",
        "    Parameters:\n",
        "    - model: Trained RNN model\n",
        "    - tokenizer: Tokenizer object used for preprocessing\n",
        "    - seed_text: Initial text to start generating from\n",
        "    - length: Number of characters to generate\n",
        "\n",
        "    Returns:\n",
        "    - generated_text: Generated text string\n",
        "    \"\"\"\n",
        "    generated_text = seed_text\n",
        "    for _ in range(length):\n",
        "        # Tokenize the input sequence\n",
        "        tokenized_sequence = tokenizer.texts_to_sequences([generated_text[-sequence_length:]])[0]\n",
        "        tokenized_sequence = pad_sequences([tokenized_sequence], maxlen=sequence_length)\n",
        "\n",
        "        # Predict next character\n",
        "        predicted_probs = model.predict(tokenized_sequence, verbose=0)[0]\n",
        "        predicted_char_index = np.argmax(predicted_probs)\n",
        "\n",
        "        # Find the character from the index\n",
        "        for char, index in tokenizer.word_index.items():\n",
        "            if index == predicted_char_index:\n",
        "                generated_text += char\n",
        "                break\n",
        "\n",
        "    return generated_text\n",
        "\n",
        "# Generate text using a seed\n",
        "seed_text = \"To be, or not to be, that is the question: \"\n",
        "generated_text = generate_text(rnn_model, tokenizer, seed_text)\n",
        "print(\"Generated Text:\\n\", generated_text)"
      ],
      "metadata": {
        "id": "1kEzps5cZ536",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "a202b7fb-75e2-4c05-bfd3-de15b0e3e620"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Generated Text:\n",
            " To be, or not to be, that is the question: if fail fail fail fail fail fail fail fail fail fail fail fail fail fail fail fail fail fail fail fail fail fail fail fail fail fail fail fail fail fail fail fail fail fail fail fail fail fail fail fa\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Exercises\n",
        "\n",
        "* Explore how different sequence lengths affect the model's performance and the coherence of the generated text.\n",
        "* Replace the simple RNN layer with a bidirectional LSTM or GRU layer to improve model performance and text generation quality."
      ],
      "metadata": {
        "id": "IqCS7NswbKY2"
      }
    }
  ]
}