05_deploying_a_chatbot_to_the_web.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "8ec2fef2",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Deploying a Chatbot to the Web\n",
    "* **Created by:** Eric Martinez\n",
    "* **For:** Software Engineering 2\n",
    "* **At:** University of Texas Rio-Grande Valley"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ffb051ff",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## HuggingFace\n",
    "\n",
    "HuggingFace is an AI research organization and platform that provides access to a wide range of pre-trained LLMs and tools for training, fine-tuning, and deploying models. It has a user-friendly interface and a large community, making it a popular choice for working with LLMs."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8b3aec8b",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Deploying to HuggingFace"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "60c8e7f6",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "Let's face it! Once we start building cool stuff we are going to want to show it off. It can take us < 10 minutes to deploy our chatbots and LLM applications when using Gradio!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7804a8ce",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "#### Configuring the files required"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "54de0ddc",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "Add a username and password for your app to your `.env` file. This will ensure that unauthorized users are not able to access LLM features. Use the following format:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "95dec7cf",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "APP_USERNAME=<whatever username you want>\n",
    "APP_PASSWORD=<whatever password you want>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5072dc21",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Let's start by taking all of our necessary chatbot code into one file which we will name `app.py`. Which I'll show you how to do in a second."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aacdcaaf",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Take note that this code has been altered a little bit from the last chatbot example in order to add authentication."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "78815586",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "In a Jupyter notebook, `%%writefile` is a magic command that tells the notebook to write the contents of the cell to a file. When you run the following cell, it will create (or overwrite) the specified file `app.py` in the current working directory and save the cell's content to that file. It won't actually run the code, just save it. This is useful when you want to save your code directly from a Jupyter notebook cell to a separate Python script file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "710b66f7",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Overwriting app.py\n"
     ]
    }
   ],
   "source": [
    "%%writefile app.py\n",
    "import gradio as gr\n",
    "import openai\n",
    "import examples as chatbot_examples\n",
    "from dotenv import load_dotenv\n",
    "import os\n",
    "\n",
    "load_dotenv()  # take environment variables from .env.\n",
    "\n",
    "# In order to authenticate, secrets must have been set, and the user supplied credentials match\n",
    "def auth(username, password):\n",
    "    app_username = os.getenv(\"APP_USERNAME\")\n",
    "    app_password = os.getenv(\"APP_PASSWORD\")\n",
    "\n",
    "    if app_username and app_password:\n",
    "        if(username == app_username and password == app_password):\n",
    "            print(\"Logged in successfully.\")\n",
    "            return True\n",
    "        else:\n",
    "            print(\"Username or password does not match.\")\n",
    "    else:\n",
    "        print(\"Credential secrets not set.\")\n",
    "    return False\n",
    "    \n",
    "# Define a function to get the AI's reply using the OpenAI API\n",
    "def get_ai_reply(message, model=\"gpt-3.5-turbo\", system_message=None, temperature=0, message_history=[]):\n",
    "    # Initialize the messages list\n",
    "    messages = []\n",
    "    \n",
    "    # Add the system message to the messages list\n",
    "    if system_message is not None:\n",
    "        messages += [{\"role\": \"system\", \"content\": system_message}]\n",
    "\n",
    "    # Add the message history to the messages list\n",
    "    if message_history is not None:\n",
    "        messages += message_history\n",
    "    \n",
    "    # Add the user's message to the messages list\n",
    "    messages += [{\"role\": \"user\", \"content\": message}]\n",
    "    \n",
    "    # Make an API call to the OpenAI ChatCompletion endpoint with the model and messages\n",
    "    completion = openai.ChatCompletion.create(\n",
    "        model=model,\n",
    "        messages=messages,\n",
    "        temperature=temperature\n",
    "    )\n",
    "    \n",
    "    # Extract and return the AI's response from the API response\n",
    "    return completion.choices[0].message.content.strip()\n",
    "\n",
    "# Define a function to handle the chat interaction with the AI model\n",
    "def chat(model, system_message, message, chatbot_messages, history_state):\n",
    "    # Initialize chatbot_messages and history_state if they are not provided\n",
    "    chatbot_messages = chatbot_messages or []\n",
    "    history_state = history_state or []\n",
    "    \n",
    "    # Try to get the AI's reply using the get_ai_reply function\n",
    "    try:\n",
    "        ai_reply = get_ai_reply(message, model=model, system_message=system_message, message_history=history_state)\n",
    "    except Exception as e:\n",
    "        # If an error occurs, raise a Gradio error\n",
    "        raise gr.Error(e)\n",
    "    \n",
    "    # Append the user's message and the AI's reply to the chatbot_messages list\n",
    "    chatbot_messages.append((message, ai_reply))\n",
    "    \n",
    "    # Append the user's message and the AI's reply to the history_state list\n",
    "    history_state.append({\"role\": \"user\", \"content\": message})\n",
    "    history_state.append({\"role\": \"assistant\", \"content\": ai_reply})\n",
    "    \n",
    "    # Return None (empty out the user's message textbox), the updated chatbot_messages, and the updated history_state\n",
    "    return None, chatbot_messages, history_state\n",
    "\n",
    "# Define a function to launch the chatbot interface using Gradio\n",
    "def get_chatbot_app(additional_examples=[]):\n",
    "    # Load chatbot examples and merge with any additional examples provided\n",
    "    examples = chatbot_examples.load_examples(additional=additional_examples)\n",
    "    \n",
    "    # Define a function to get the names of the examples\n",
    "    def get_examples():\n",
    "        return [example[\"name\"] for example in examples]\n",
    "\n",
    "    # Define a function to choose an example based on the index\n",
    "    def choose_example(index):\n",
    "        if(index!=None):\n",
    "            system_message = examples[index][\"system_message\"].strip()\n",
    "            user_message = examples[index][\"message\"].strip()\n",
    "            return system_message, user_message, [], []\n",
    "        else:\n",
    "            return \"\", \"\", [], []\n",
    "\n",
    "    # Create the Gradio interface using the Blocks layout\n",
    "    with gr.Blocks() as app:\n",
    "        with gr.Tab(\"Conversation\"):\n",
    "            with gr.Row():\n",
    "                with gr.Column():\n",
    "                    # Create a dropdown to select examples\n",
    "                    example_dropdown = gr.Dropdown(get_examples(), label=\"Examples\", type=\"index\")\n",
    "                    # Create a button to load the selected example\n",
    "                    example_load_btn = gr.Button(value=\"Load\")\n",
    "                    # Create a textbox for the system message (prompt)\n",
    "                    system_message = gr.Textbox(label=\"System Message (Prompt)\", value=\"You are a helpful assistant.\")\n",
    "                with gr.Column():\n",
    "                    # Create a dropdown to select the AI model\n",
    "                    model_selector = gr.Dropdown(\n",
    "                        [\"gpt-3.5-turbo\"],\n",
    "                        label=\"Model\",\n",
    "                        value=\"gpt-3.5-turbo\"\n",
    "                    )\n",
    "                    # Create a chatbot interface for the conversation\n",
    "                    chatbot = gr.Chatbot(label=\"Conversation\")\n",
    "                    # Create a textbox for the user's message\n",
    "                    message = gr.Textbox(label=\"Message\")\n",
    "                    # Create a state object to store the conversation history\n",
    "                    history_state = gr.State()\n",
    "                    # Create a button to send the user's message\n",
    "                    btn = gr.Button(value=\"Send\")\n",
    "\n",
    "                # Connect the example load button to the choose_example function\n",
    "                example_load_btn.click(choose_example, inputs=[example_dropdown], outputs=[system_message, message, chatbot, history_state])\n",
    "                # Connect the send button to the chat function\n",
    "                btn.click(chat, inputs=[model_selector, system_message, message, chatbot, history_state], outputs=[message, chatbot, history_state])\n",
    "        # Return the app\n",
    "        return app\n",
    "        \n",
    "# Call the launch_chatbot function to start the chatbot interface using Gradio\n",
    "# Set the share parameter to False, meaning the interface will not be publicly accessible\n",
    "app = get_chatbot_app()\n",
    "app.queue()  # this is to be able to queue multiple requests at once\n",
    "app.launch(auth=auth)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6d75af66",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "We will also need a `requirements.txt` file to store the list of the packages that HuggingFace needs to install to run our chatbot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "14d0e434",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Overwriting requirements.txt\n"
     ]
    }
   ],
   "source": [
    "%%writefile requirements.txt\n",
    "gradio == 3.27.0\n",
    "openai == 0.27.4\n",
    "python-dotenv == 1.0.0"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4debec45",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Now let's go ahead and commit our changes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "14d42a96",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "!git add app.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "d7c5b127",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "!git add requirements.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "18960d9f",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[master (root-commit) 39899ef] adding chatbot\r\n",
      " 2 files changed, 101 insertions(+)\r\n",
      " create mode 100644 app.py\r\n",
      " create mode 100644 requirements.txt\r\n"
     ]
    }
   ],
   "source": [
    "!git commit -m \"adding chatbot\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "09221ee0",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "#### Using HuggingFace Spaces"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "db789c94",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "As mentioned before, HuggingFace is a free-to-use platform for hosting AI demos and apps. We will need to make a HuggingFace _Space_ for our chatbot."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d9eedd10",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "First sign up for a free HuggingFace account [here](https://huggingface.co/join). "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "00f537d9",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "<img src=\"images/signup.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f5fc871f",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "After you sign up, create a new Space by clicking \"New Space\" on the navigation menu (press on your profile image)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f84e8451",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "<img src=\"images/dropdown.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "85684a7b",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Create a new space with the following settings"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e6130b5d",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "<img src=\"images/space-settings.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "897729d3",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Take note of your HuggingFace Space URL"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3e042d24",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "#### Generate a HuggingFace Access Token"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "67273b1a",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "In order to push our code to HuggingFace, we will need to authenticate with HuggingFace via an access token on Git.\n",
    "\n",
    "HuggingFace uses git to push a copy of your code to their servers for deployment."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "909216d4",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Visit your Profile Settings and click 'Access Tokens'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fc1c1bec",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "<img src=\"images/profile-settings.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4af1eb8e",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Create a new access token for your app with 'write' permissions."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e4958b80",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "<img src=\"images/access-tokens.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "230d1a32",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "<img src=\"images/new-access-token.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f050684c",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Copy the token to your clipboard!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aa776b02",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "<img src=\"images/copy-token.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a7d0781d",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "#### Login to HuggingFace Hub"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "74518b91",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "If you have ever used Heroku, HuggingFace is similar. The way you deploy is by pushing them a copy of your git repository.\n",
    "\n",
    "In order for us to do this, our local git will need to be able to authenticate with HuggingFace.\n",
    "\n",
    "Fortunately, theres an easy way we can authenticate and we can do it from inside this notebook."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eba83252",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Install `huggingface_hub`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "266bf481",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [],
   "source": [
    "!pip -q install --upgrade huggingface_hub"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d50cd84b",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Login to HuggingFace"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "53fd5037",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Token is valid.\n",
      "Your token has been saved in your configured git credential helpers (osxkeychain).\n",
      "Your token has been saved to /Users/ericmartinez/.cache/huggingface/token\n",
      "Login successful\n"
     ]
    }
   ],
   "source": [
    "from huggingface_hub import notebook_login\n",
    "notebook_login()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "90f9bd4d",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "#### Now lets setup git and HuggingFace Spaces to work together and deploy"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "66468481",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "<span style=\"color:red\">REPLACE MY URL WITH YOURS</span>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "827a201d",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [],
   "source": [
    "!git remote add huggingface https://huggingface.co/spaces/ericmichael/gradio-chatbot-demo"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f8b3bb3d",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "Then force push to sync everything for the first time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "86c9ee4e",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total 0 (delta 0), reused 0 (delta 0), pack-reused 0\n",
      "To https://huggingface.co/spaces/ericmichael/gradio-chatbot-demo\n",
      " + 8911ec0...3693bcc main -> main (forced update)\n"
     ]
    }
   ],
   "source": [
    "!git push --force huggingface main"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "48510bba",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "#### Now lets setup our secrets on our HuggingFace Space"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4ab2d59b",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "We're so close, HuggingFace now has our app but it doesn't have any of our secrets! This means users can't authenticate with our HuggingFace app and our app can't authenticate with OpenAI!\n",
    "\n",
    "We will need to set our secrets manually on HuggingFace, which is a common practice so they don't end up accidentally leaked in version control."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fec397c8",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Go to your HuggingFace Space and click 'Settings'"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a7016df7",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "<img src=\"images/repo.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1e10cf24",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Scroll to 'Repository Secrets', yours will be empty for now but you will need to fill it in."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d70ec1aa",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "<img src=\"images/secrets.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8c931425",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "We will need to provide:\n",
    "\n",
    "* `OPENAI_API_BASE` - if you are using my OpenAI service, don't add this one if you are paying for OpenAI yourself\n",
    "* `OPENAI_API_KEY` - either the key from my service or from OpenAI directly\n",
    "* `APP_USERNAME` - the username you want people to sign in with\n",
    "* `APP_PASSWORD` - the username you want people to sign in with"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "17a2e7c5",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Example"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7132a08f",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "<img src=\"images/add-secret.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4405e200",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "Restart your HuggingFace Space"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0718c510",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "<img src=\"images/restart.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3a353ebf",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "source": [
    "That's it! 🎉 Check your HuggingFace Space URL to access your chatbot!"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}