update notebooks

01_introduction_to_llms.ipynb

@@ -9,7 +9,7 @@
     }
    },
    "source": [
-    "# Lecture 1: Introduction to Large Language Models\n",
+    "# Introduction to Large Language Models\n",
     "* **Created by:** Eric Martinez\n",
     "* **For:** Software Engineering 2\n",
     "* **At:** University of Texas Rio-Grande Valley"

02_prototyping_a_basic_chatbot_ui.ipynb

@@ -0,0 +1,258 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "8ec2fef2",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
+   "source": [
+    "# Prototyping a Basic Chatbot UI\n",
+    "* **Created by:** Eric Martinez\n",
+    "* **For:** Software Engineering 2\n",
+    "* **At:** University of Texas Rio-Grande Valley"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "e989bbe3",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
+   "source": [
+    "## Tools and Concepts"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "368cda1a",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
+   "source": [
+    "#### Gradio\n",
+    "\n",
+    "Gradio is an open-source Python library that allows developers to create and prototype user interfaces (UIs) and APIs for machine learning applications and LLMs quickly and easily. Three reasons it is a great tool are:\n",
+    "\n",
+    "* Simple and intuitive: Gradio makes it easy to create UIs with minimal code, allowing developers to focus on their models.\n",
+    "* Versatile: Gradio supports a wide range of input and output types, making it suitable for various machine learning tasks.\n",
+    "* Shareable: Gradio interfaces can be shared with others, enabling collaboration and easy access to your models."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "e2888a24",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
+   "source": [
+    "#### Chatbots\n",
+    "\n",
+    "Chatbots are AI-powered conversational agents designed to interact with users through natural language. LLMs have the potential to revolutionize chatbots by providing more human-like, accurate, and context-aware responses, leading to more engaging and useful interactions."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "6f24bac1",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
+   "source": [
+    "#### Example: Gradio Interface\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "4ecdb50a",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
+   "outputs": [],
+   "source": [
+    "!pip -q install --upgrade gradio"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "0b28a4e7",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
+   "outputs": [],
+   "source": [
+    "import gradio as gr\n",
+    " \n",
+    "def do_something_cool(first_name, last_name):\n",
+    "    return f\"{first_name} {last_name} is so cool\"\n",
+    "    \n",
+    "with gr.Blocks() as app:\n",
+    "        first_name_box = gr.Textbox(label=\"First Name\")\n",
+    "        last_name_box = gr.Textbox(label=\"Last Name\")\n",
+    "        output_box = gr.Textbox(label=\"Output\", interactive=False)\n",
+    "        btn = gr.Button(value =\"Send\")\n",
+    "        btn.click(do_something_cool, inputs = [first_name_box, last_name_box], outputs = [output_box])\n",
+    "    app.launch(share=True)\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "c1b1b69e",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
+   "source": [
+    "#### Example: Basic Gradio Chatbot Interface / Just the look"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "ea2878bb",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
+   "source": [
+    "Version with minimal comments"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "a6f1cca3",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
+   "outputs": [],
+   "source": [
+    "import gradio as gr\n",
+    "\n",
+    "def chat(message, history):\n",
+    "    history = history or []\n",
+    "    # Set a simple AI reply (replace this with a call to an LLM for a more sophisticated response)\n",
+    "    ai_reply = \"hello\"      \n",
+    "    history.append((message, ai_reply))\n",
+    "    return None, history, history\n",
+    "    \n",
+    "with gr.Blocks() as app:\n",
+    "    with gr.Tab(\"Conversation\"):\n",
+    "        with gr.Row():\n",
+    "            with gr.Column():\n",
+    "                chatbot = gr.Chatbot(label=\"Conversation\")\n",
+    "                message = gr.Textbox(label=\"Message\")\n",
+    "                history_state = gr.State()\n",
+    "                btn = gr.Button(value =\"Send\")\n",
+    "            btn.click(chat, inputs = [message, history_state], outputs = [message, chatbot, history_state])\n",
+    "    app.launch(share=True)\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "5cc72efb",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
+   "source": [
+    "Version with more comments"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "44debca6",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
+   "outputs": [],
+   "source": [
+    "import gradio as gr\n",
+    "\n",
+    "# Define the chat function that processes user input and generates an AI response\n",
+    "def chat(message, history):\n",
+    "    # If history is empty, initialize it as an empty list\n",
+    "    history = history or []\n",
+    "    # Set a simple AI reply (replace this with a call to an LLM for a more sophisticated response)\n",
+    "    ai_reply = \"hello\"\n",
+    "    # Append the user message and AI reply to the conversation history\n",
+    "    history.append((message, ai_reply))\n",
+    "    # Return the updated history and display it in the chatbot interface\n",
+    "    return None, history, history\n",
+    "\n",
+    "# Create a Gradio Blocks interface\n",
+    "with gr.Blocks() as app:\n",
+    "    # Create a tab for the conversation\n",
+    "    with gr.Tab(\"Conversation\"):\n",
+    "        # Create a row for the input components\n",
+    "        with gr.Row():\n",
+    "            # Create a column for the input components\n",
+    "            with gr.Column():\n",
+    "                # Create a chatbot component to display the conversation\n",
+    "                chatbot = gr.Chatbot(label=\"Conversation\")\n",
+    "                # Create a textbox for user input\n",
+    "                message = gr.Textbox(label=\"Message\")\n",
+    "                # Create a state variable to store the conversation history\n",
+    "                history_state = gr.State()\n",
+    "                # Create a button to send the user's message\n",
+    "                btn = gr.Button(value=\"Send\")\n",
+    "            # Connect the button click event to the chat function, passing in the input components and updating the output components\n",
+    "            btn.click(chat, inputs=[message, history_state], outputs=[message, chatbot, history_state])\n",
+    "    # Launch the Gradio interface and make it shareable\n",
+    "    app.launch(share=True)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "3b660170",
+   "metadata": {},
+   "source": [
+    "Sweet! That wasn't too bad, not much code to make a cool little UI!"
+   ]
+  }
+ ],
+ "metadata": {
+  "celltoolbar": "Raw Cell Format",
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.8"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

03_using_the_openai_api.ipynb

@@ -0,0 +1,784 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "8ec2fef2",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
+   "source": [
+    "# Using the OpenAI API\n",
+    "* **Created by:** Eric Martinez\n",
+    "* **For:** Software Engineering 2\n",
+    "* **At:** University of Texas Rio-Grande Valley"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "6c3f79e3",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
+   "source": [
+    "## OpenAI API\n",
+    "\n",
+    "The OpenAI API provides access to powerful LLMs like GPT-3.5 and GPT-4, enabling developers to leverage these models in their applications. To access the API, sign up for an API key on the OpenAI website and follow the documentation to make API calls.\n",
+    "\n",
+    "For enterprise: Azure OpenAI offers a robust and scalable platform for deploying LLMs in enterprise applications. It provides features like security, compliance, and support, making it an ideal choice for businesses looking to leverage LLMs.\n",
+    "\n",
+    "Options:\n",
+    "* [[Free] Sign-up for access to my OpenAI service](https://platform.openai.com/signup) - _requires your UTRGV email and student ID_\n",
+    "* [[Paid] Alternatively, sign-up for OpenAI API Access](https://platform.openai.com/signup)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "c412a4e9",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
+   "source": [
+    "## Managing Application Secrets"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "66de4ac1",
+   "metadata": {},
+   "source": [
+    "Secrets are sensitive information, such as API keys, passwords, or cryptographic keys, that must be protected to ensure the security and integrity of a system.\n",
+    "\n",
+    "In software development, secrets are often used to authenticate users, grant access to resources, or encrypt/decrypt data. Mismanaging or exposing secrets can lead to severe security breaches and data leaks.\n",
+    "\n",
+    "Common examples of secrets\n",
+    "* API keys\n",
+    "* Database credentials\n",
+    "* SSH keys\n",
+    "* OAuth access tokens\n",
+    "* Encryption/decryption keys\n",
+    "\n",
+    "Common mistakes when handling secrets\n",
+    "* Storing secrets in plain text\n",
+    "* Hardcoding secrets in source code\n",
+    "* Sharing secrets through unsecured channels (e.g., email or messaging apps)\n",
+    "* Using the same secret for multiple purposes\n",
+    "* Not rotating or updating secrets regularly\n",
+    "\n",
+    "How attackers might obtain secrets\n",
+    "* Exploiting vulnerabilities in software or infrastructure\n",
+    "* Intercepting unencrypted communications\n",
+    "* Gaining unauthorized access to repositories or storage systems\n",
+    "* Social engineering or phishing attacks\n",
+    "* Brute-forcing weak secrets\n",
+    "\n",
+    "The impact of compromised secrets\n",
+    "* Unauthorized access to sensitive data\n",
+    "* Data breaches, resulting in financial loss and reputational damage\n",
+    "* Loss of trust from customers and stakeholders\n",
+    "* Legal repercussions and regulatory fines\n",
+    "* Potential takeover or manipulation of systems\n",
+    "\n",
+    "Steps to protect secrets\n",
+    "* Store secrets securely using secret management tools or dedicated secret storage services\n",
+    "* Encrypt secrets at rest and in transit\n",
+    "* Use strong, unique secrets and rotate them regularly\n",
+    "* Limit access to secrets on a need-to-know basis\n",
+    "* Implement proper auditing and monitoring of secret usage\n",
+    "\n",
+    " Cloud services and secret management\n",
+    "* Many cloud providers offer secret management services (e.g., AWS Secrets Manager, Azure Key Vault, Google Cloud Secret Manager) that securely store, manage, and rotate secrets.\n",
+    "* These services often provide access control, encryption, and auditing capabilities.\n",
+    "* Integrating cloud secret management services with your application can help secure sensitive information and reduce the risk of exposure.\n",
+    "\n",
+    "Best practices for secrets\n",
+    "* Use different secrets for development, testing, and production environments to minimize the risk of accidental exposure.\n",
+    "* Regularly audit and review secret access to ensure only authorized users have access.\n",
+    "* Establish a clear process for managing secrets, including secret creation, storage, rotation, and deletion.\n",
+    "* Educate team members on the importance of secret security and the best practices for handling sensitive information."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "ef366b65",
+   "metadata": {},
+   "source": [
+    "#### Using  `.dotenv` library to protect secrets in Python"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "dc39df10",
+   "metadata": {},
+   "source": [
+    " `.dotenv` is a Python library that allows developers to load environment variables from a `.env` file. It helps keep secrets out of source code and makes it easier to manage and update them."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "88252d32",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "      3. Add secrets as key-value pairs in the `.env` file\n",
+    "      4. Load secrets in your Python code using the `load_dotenv()` function\n",
+    "      5. Access secrets using `os.environ`"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "ae0500ea",
+   "metadata": {},
+   "source": [
+    "Install the `python-dotenv` library"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 45,
+   "id": "1212333f",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!pip -q install --upgrade python-dotenv"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "faecedf0",
+   "metadata": {},
+   "source": [
+    "Create a `.env` file in this folder using any editor."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "a880382a",
+   "metadata": {},
+   "source": [
+    "Add secrets as key-value pairs in the `.env` file"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "04f00703",
+   "metadata": {},
+   "source": [
+    "If you are using my OpenAI service use the following format:"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "70e183f5",
+   "metadata": {},
+   "source": [
+    "OPENAI_API_BASE=<my API base>\n",
+    "OPENAI_API_KEY=<your API key to my service>"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "a952b103",
+   "metadata": {},
+   "source": [
+    "If you are not using my OpenAI service then use the following format:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "7cf6ed7b",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "OPENAI_API_KEY=<your OpenAI API key>"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "955963ed",
+   "metadata": {},
+   "source": [
+    "Then, use the following code to load those secrets into this notebook:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 47,
+   "id": "fcadf45e",
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "True"
+      ]
+     },
+     "execution_count": 47,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "from dotenv import load_dotenv\n",
+    "\n",
+    "load_dotenv()  # take environment variables from .env."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "d3b6c394",
+   "metadata": {},
+   "source": [
+    "#### Install Dependencies"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "id": "bcc79375",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
+   "outputs": [],
+   "source": [
+    "!pip -q install --upgrade openai"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "f2ed966d",
+   "metadata": {},
+   "source": [
+    "#### Let's make a function to wrap OpenAI functionality and write some basic tests"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "c1b09026",
+   "metadata": {},
+   "source": [
+    "Start by simply seeing if we can make an API call"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "0abdd4e9",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Hello there! How may I assist you today?\n"
+     ]
+    }
+   ],
+   "source": [
+    "import openai\n",
+    "from dotenv import load_dotenv\n",
+    "\n",
+    "load_dotenv()  # take environment variables from .env.\n",
+    "\n",
+    "model=\"gpt-3.5-turbo\"\n",
+    "messages=[{\"role\": \"user\", \"content\": \"hello\"}]\n",
+    "\n",
+    "completion = openai.ChatCompletion.create(model=model, messages=messages)\n",
+    "\n",
+    "print(completion.choices[0].message.content)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "3f5d1530",
+   "metadata": {},
+   "source": [
+    "Great! Now let's wrap that in a function"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 19,
+   "id": "af4895c3",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Hello! How can I assist you today?\n"
+     ]
+    }
+   ],
+   "source": [
+    "import openai\n",
+    "from dotenv import load_dotenv\n",
+    "\n",
+    "load_dotenv()  # take environment variables from .env.\n",
+    "\n",
+    "def get_ai_reply(model=\"gpt-3.5-turbo\", user_message=\"\"):\n",
+    "    messages=[{\"role\": \"user\", \"content\": user_message}]\n",
+    "    completion = openai.ChatCompletion.create(model=model, messages=messages)\n",
+    "    return completion.choices[0].message.content\n",
+    "\n",
+    "print(get_ai_reply(user_message=\"hello\"))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "f4506fe8",
+   "metadata": {},
+   "source": [
+    "Let's add some tests!\n",
+    "\n",
+    "These are traditional type of tests.\n",
+    "\n",
+    "Since the output is non-deterministic, generally, what are some things that we could test?\n",
+    "\n",
+    "At the very least, maybe that the output is a string?"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 20,
+   "id": "42a36d3a",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
+   "outputs": [],
+   "source": [
+    "import openai\n",
+    "from dotenv import load_dotenv\n",
+    "\n",
+    "load_dotenv()  # take environment variables from .env.\n",
+    "\n",
+    "def get_ai_reply(message, model=\"gpt-3.5-turbo\"):    \n",
+    "    messages=[{\"role\": \"user\", \"content\": message}]\n",
+    "        \n",
+    "    completion = openai.ChatCompletion.create(model=model, messages=messages, temperature=temperature)\n",
+    "    return completion.choices[0].message.content\n",
+    "\n",
+    "# traditional tests\n",
+    "assert isinstance(get_ai_reply(\"hello\"), str)\n",
+    "assert isinstance(get_ai_reply(\"hello\", model=\"gpt-3.5-turbo\"), str)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "ed166c5b",
+   "metadata": {},
+   "source": [
+    "But what if we do what to test the output of the LLM?\n",
+    "\n",
+    "Is there anyway to control, atleast to some degree, the level of non-determinism?\n",
+    "\n",
+    "Yes! Let's add a temperature parameter, this will help us control the 'creativity' and 'randomness' of the response.\n",
+    "\n",
+    "Setting it to 0 helps ensure outputs are more consistent when given the same input.\n",
+    "\n",
+    "Valid values for temperature are between 0 and 1, inclusive.\n",
+    "\n",
+    "This will help us when writing tests, but is something that we should keep in mind that if we write tests against the LLM output, we might get the expect results _ONLY_ at low temperature.\n",
+    "\n",
+    "An ideal test strategy should resemble the temperature setting we will use in production."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 31,
+   "id": "9c03b774",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import openai\n",
+    "from dotenv import load_dotenv\n",
+    "\n",
+    "load_dotenv()  # take environment variables from .env.\n",
+    "\n",
+    "def get_ai_reply(message, model=\"gpt-3.5-turbo\", temperature=0):    \n",
+    "    messages=[{\"role\": \"user\", \"content\": message}]\n",
+    "        \n",
+    "    completion = openai.ChatCompletion.create(model=model, messages=messages, temperature=temperature)\n",
+    "    return completion.choices[0].message.content\n",
+    "\n",
+    "# traditional tests\n",
+    "assert isinstance(get_ai_reply(\"hello\"), str)\n",
+    "assert isinstance(get_ai_reply(\"hello\", model=\"gpt-3.5-turbo\"), str)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 33,
+   "id": "f5ed24da",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Hello! How can I assist you today?\n",
+      "Hello there! How can I assist you today?\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(get_ai_reply(\"hello\")) # uses default of temperature 0\n",
+    "print(get_ai_reply(\"hello\", temperature=0.9)) # uses default of temperature 0"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "ee320648",
+   "metadata": {},
+   "source": [
+    "Ok great! Now, an LLM is no good to us if we can't _steer_ it.\n",
+    "\n",
+    "So let's add the ability to input a _prompt_ or _system message_."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 37,
+   "id": "295839a5",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import openai\n",
+    "from dotenv import load_dotenv\n",
+    "\n",
+    "load_dotenv()  # take environment variables from .env.\n",
+    "\n",
+    "# Define a function to get the AI's reply using the OpenAI API\n",
+    "def get_ai_reply(message, model=\"gpt-3.5-turbo\", system_message=None, temperature=0):\n",
+    "    # Initialize the messages list\n",
+    "    messages = []\n",
+    "    \n",
+    "    # Add the system message to the messages list\n",
+    "    if system_message is not None:\n",
+    "        messages += [{\"role\": \"system\", \"content\": system_message}]\n",
+    "    \n",
+    "    # Add the user's message to the messages list\n",
+    "    messages += [{\"role\": \"user\", \"content\": message}]\n",
+    "    \n",
+    "    # Make an API call to the OpenAI ChatCompletion endpoint with the model and messages\n",
+    "    completion = openai.ChatCompletion.create(\n",
+    "        model=model,\n",
+    "        messages=messages,\n",
+    "        temperature=temperature\n",
+    "    )\n",
+    "    \n",
+    "    # Extract and return the AI's response from the API response\n",
+    "    return completion.choices[0].message.content.strip()\n",
+    "\n",
+    "# traditional tests\n",
+    "assert isinstance(get_ai_reply(\"hello\"), str)\n",
+    "assert isinstance(get_ai_reply(\"hello\", model=\"gpt-3.5-turbo\"), str)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "fc9763d6",
+   "metadata": {},
+   "source": [
+    "Let's see if we can get the LLM to follow instructions by adding instructions to the prompt.\n",
+    "\n",
+    "Run this cell a few times and see what happens. Is it consistent?"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 36,
+   "id": "c5e2a8b3",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "world.\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(get_ai_reply(\"hello\", model=\"gpt-3.5-turbo\", system_message=\"When I say 'hello' you simply reply with 'world.'\"))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "e59d3275",
+   "metadata": {},
+   "source": [
+    "While the output is more or less controlled, the LLM responds with 'world.' or 'world'. While the word 'world' being in the string is pretty consistent, the punctuation is not.\n",
+    "\n",
+    "We could do additional tweaking to the prompt to prevent that if we wanted.\n",
+    "\n",
+    "For now let's assume this is the best we can do.\n",
+    "\n",
+    "How do we write tests against this?\n",
+    "\n",
+    "Well, what do have high confidence won't change in the LLM output?\n",
+    "\n",
+    "What is a test that we could write that:\n",
+    "* would pass if the LLM outputs in a manner that is consistent with our expectations (and consistent with its own output)?\n",
+    "* _we want to be true_ about our LLM system, and if it does not then we would want to know immediately and adjust our system?\n",
+    "* if the prompt does not change, that our expectation holds true?\n",
+    "* someone changes the prompt in a way that would break the rest of the system, that we would want to prevent that from being merged without fixing the downstream effects?\n",
+    "\n",
+    "That might be a bunch of ways of saying the same thing but I hope you get the point."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 38,
+   "id": "d2b2bc15",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# non-deterministic tests\n",
+    "system_message=\"When I say 'hello' you simply reply with 'world.'\"\n",
+    "message=\"hello\"\n",
+    "\n",
+    "assert \"world\" in get_ai_reply(message, system_message=system_message)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "5e17eefe",
+   "metadata": {},
+   "source": [
+    "Alright that worked!\n",
+    "\n",
+    "Now, let's extend the functionality to add the ability to pass message history so that it can have memory about what was said previously."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 39,
+   "id": "4fd88c05",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import openai\n",
+    "from dotenv import load_dotenv\n",
+    "\n",
+    "load_dotenv()  # take environment variables from .env.\n",
+    "\n",
+    "# Define a function to get the AI's reply using the OpenAI API\n",
+    "def get_ai_reply(message, model=\"gpt-3.5-turbo\", system_message=None, temperature=0, message_history=[]):\n",
+    "    # Initialize the messages list\n",
+    "    messages = []\n",
+    "    \n",
+    "    # Add the system message to the messages list\n",
+    "    if system_message is not None:\n",
+    "        messages += [{\"role\": \"system\", \"content\": system_message}]\n",
+    "    \n",
+    "    # Add the message history to the messages list\n",
+    "    if message_history is not None:\n",
+    "        messages += message_history\n",
+    "    \n",
+    "    # Add the user's message to the messages list\n",
+    "    messages += [{\"role\": \"user\", \"content\": message}]\n",
+    "    \n",
+    "    # Make an API call to the OpenAI ChatCompletion endpoint with the model and messages\n",
+    "    completion = openai.ChatCompletion.create(\n",
+    "        model=model,\n",
+    "        messages=messages,\n",
+    "        temperature=temperature\n",
+    "    )\n",
+    "    \n",
+    "    # Extract and return the AI's response from the API response\n",
+    "    return completion.choices[0].message.content.strip()\n",
+    "\n",
+    "# traditional tests\n",
+    "assert isinstance(get_ai_reply(\"hello\"), str)\n",
+    "assert isinstance(get_ai_reply(\"hello\", model=\"gpt-3.5-turbo\"), str)\n",
+    "\n",
+    "# non-deterministic unit tests\n",
+    "assert \"world\" in get_ai_reply(\"hello\", model=\"gpt-3.5-turbo\", system_message=\"When I say 'hello' you simply reply with 'world.'\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "e0a00cda",
+   "metadata": {},
+   "source": [
+    "Now let's check that it works"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 42,
+   "id": "977f99bd",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Your name is Bob.\n"
+     ]
+    }
+   ],
+   "source": [
+    "system_message=\"The user will tell you their name. When asked, repeat their name back to them.\"\n",
+    "message_history = [\n",
+    "   {\"role\": \"user\", \"content\": \"My name is Bob.\"},\n",
+    "   {\"role\": \"assistant\", \"content\": \"Nice to meet you, Bob.\"}\n",
+    "]\n",
+    "message = \"What was my name again?\"\n",
+    "print(get_ai_reply(message, system_message=system_message, message_history=message_history))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "2ad45888",
+   "metadata": {},
+   "source": [
+    "Great! Now let's turn that into a test!"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 43,
+   "id": "83aa7546",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "system_message=\"The user will tell you their name. When asked, repeat their name back to them.\"\n",
+    "message_history = [\n",
+    "   {\"role\": \"user\", \"content\": \"My name is Bob.\"},\n",
+    "   {\"role\": \"assistant\", \"content\": \"Nice to meet you, Bob.\"}\n",
+    "]\n",
+    "message = \"What was my name again?\"\n",
+    "\n",
+    "assert \"Bob\" in get_ai_reply(message, system_message=system_message, message_history=message_history)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "25e498c8",
+   "metadata": {},
+   "source": [
+    "Alright here is our final function for integrating with OpenAI!"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "bfc5cd86",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import openai\n",
+    "from dotenv import load_dotenv\n",
+    "\n",
+    "load_dotenv()  # take environment variables from .env.\n",
+    "\n",
+    "# Define a function to get the AI's reply using the OpenAI API\n",
+    "def get_ai_reply(message, model=\"gpt-3.5-turbo\", system_message=None, temperature=0, message_history=[]):\n",
+    "    # Initialize the messages list\n",
+    "    messages = []\n",
+    "    \n",
+    "    # Add the system message to the messages list\n",
+    "    if system_message is not None:\n",
+    "        messages += [{\"role\": \"system\", \"content\": system_message}]\n",
+    "\n",
+    "    # Add the message history to the messages list\n",
+    "    if message_history is not None:\n",
+    "        messages += message_history\n",
+    "    \n",
+    "    # Add the user's message to the messages list\n",
+    "    messages += [{\"role\": \"user\", \"content\": message}]\n",
+    "    \n",
+    "    # Make an API call to the OpenAI ChatCompletion endpoint with the model and messages\n",
+    "    completion = openai.ChatCompletion.create(\n",
+    "        model=model,\n",
+    "        messages=messages,\n",
+    "        temperature=temperature\n",
+    "    )\n",
+    "    \n",
+    "    # Extract and return the AI's response from the API response\n",
+    "    return completion.choices[0].message.content.strip()\n",
+    "\n",
+    "# traditional unit tests\n",
+    "assert isinstance(get_ai_reply(\"hello\"), str)\n",
+    "assert isinstance(get_ai_reply(\"hello\", model=\"gpt-3.5-turbo\"), str)\n",
+    "\n",
+    "# non-deterministic unit tests\n",
+    "assert \"world\" in get_ai_reply(\"hello\", model=\"gpt-3.5-turbo\", system_message=\"When I say 'hello' you simply reply with 'world.'\")\n",
+    "\n",
+    "system_message=\"The user will tell you their name. When asked, repeat their name back to them.\"\n",
+    "message_history = [\n",
+    "   {\"role\": \"user\", \"content\": \"My name is Bob.\"},\n",
+    "   {\"role\": \"assistant\", \"content\": \"Nice to meet you, Bob.\"}\n",
+    "]\n",
+    "message = \"What was my name again?\"\n",
+    "\n",
+    "assert \"Bob\" in get_ai_reply(message, system_message=system_message, message_history=message_history)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 44,
+   "id": "159eea8a",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Hello! How can I assist you today?\n"
+     ]
+    }
+   ],
+   "source": [
+    "print(get_ai_reply(\"hello\"))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "9de8f5da",
+   "metadata": {},
+   "source": [
+    "In the next few lessons, we will be building a graphical user interface around this functionality so we can have a real conversational experience."
+   ]
+  }
+ ],
+ "metadata": {
+  "celltoolbar": "Raw Cell Format",
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.8"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

04_creating_a_functional_conversational_chatbot.ipynb

@@ -0,0 +1,380 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "id": "8ec2fef2",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
+   "source": [
+    "# Creating a Functional Conversational Chatbot\n",
+    "* **Created by:** Eric Martinez\n",
+    "* **For:** Software Engineering 2\n",
+    "* **At:** University of Texas Rio-Grande Valley"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "cd292e78",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
+   "source": [
+    "## Tutorial: A Basic Conversational Chatbot with LLM (has limitations)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "dce44841",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
+   "source": [
+    "#### Installing Dependencies"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 8,
+   "id": "1bae55e3",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
+   "outputs": [],
+   "source": [
+    "!pip -q install --upgrade gradio\n",
+    "!pip -q install --upgrade openai"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "f762bbca",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
+   "source": [
+    "#### Creating a basic Chatbot UI using Gradio"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 9,
+   "id": "feb92318",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Running on local URL:  http://127.0.0.1:7875\n",
+      "Running on public URL: https://8b8aa5ec7c7f25f014.gradio.live\n",
+      "\n",
+      "This share link expires in 72 hours. For free permanent hosting and GPU upgrades (NEW!), check out Spaces: https://huggingface.co/spaces\n"
+     ]
+    },
+    {
+     "data": {
+      "text/html": [
+       "<div><iframe src=\"https://8b8aa5ec7c7f25f014.gradio.live\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "import gradio as gr\n",
+    "import openai\n",
+    "from dotenv import load_dotenv\n",
+    "\n",
+    "load_dotenv()  # take environment variables from .env.\n",
+    "\n",
+    "# Define a function to get the AI's reply using the OpenAI API\n",
+    "def get_ai_reply(message, model=\"gpt-3.5-turbo\", system_message=None, temperature=0, message_history=[]):\n",
+    "    # Initialize the messages list\n",
+    "    messages = []\n",
+    "    \n",
+    "    # Add the system message to the messages list\n",
+    "    if system_message is not None:\n",
+    "        messages += [{\"role\": \"system\", \"content\": system_message}]\n",
+    "\n",
+    "    # Add the message history to the messages list\n",
+    "    if message_history is not None:\n",
+    "        messages += message_history\n",
+    "    \n",
+    "    # Add the user's message to the messages list\n",
+    "    messages += [{\"role\": \"user\", \"content\": message}]\n",
+    "    \n",
+    "    # Make an API call to the OpenAI ChatCompletion endpoint with the model and messages\n",
+    "    completion = openai.ChatCompletion.create(\n",
+    "        model=model,\n",
+    "        messages=messages,\n",
+    "        temperature=temperature\n",
+    "    )\n",
+    "    \n",
+    "    # Extract and return the AI's response from the API response\n",
+    "    return completion.choices[0].message.content.strip()\n",
+    "\n",
+    "def chat(message, history):\n",
+    "    history = history or []\n",
+    "    ai_reply = get_ai_reply(message)       \n",
+    "    history.append((message, ai_reply))\n",
+    "    return None, history, history\n",
+    "    \n",
+    "with gr.Blocks() as demo:\n",
+    "    with gr.Tab(\"Conversation\"):\n",
+    "        with gr.Row():\n",
+    "            with gr.Column():\n",
+    "                chatbot = gr.Chatbot(label=\"Conversation\")\n",
+    "                message = gr.Textbox(label=\"Message\")\n",
+    "                history_state = gr.State()\n",
+    "                btn = gr.Button(value =\"Send\")\n",
+    "            btn.click(chat, inputs = [message, history_state], outputs = [message, chatbot, history_state])\n",
+    "    demo.launch(share=True)\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "77c9ec20",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
+   "source": [
+    "#### Limitations\n",
+    "* Hardcoded to 'gpt-3.5-turbo' in the UI\n",
+    "* No error-handling on the API request\n",
+    "* While the OpenAI function takes message history, the UI doesn't pass it through\n",
+    "* Doesn't use or allow prompt or 'system' message customization"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "66215904",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
+   "source": [
+    "## Tutorial: Improved Chatbot"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "99f57faf",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
+   "source": [
+    "The following snippet adds conversation history to the Gradio chat functionality, handles errors, and passes along the system message."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 27,
+   "id": "9e55e844",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
+   "outputs": [],
+   "source": [
+    "# Define a function to handle the chat interaction with the AI model\n",
+    "def chat(model, system_message, message, chatbot_messages, history_state):\n",
+    "    # Initialize chatbot_messages and history_state if they are not provided\n",
+    "    chatbot_messages = chatbot_messages or []\n",
+    "    history_state = history_state or []\n",
+    "    \n",
+    "    # Try to get the AI's reply using the get_ai_reply function\n",
+    "    try:\n",
+    "        ai_reply = get_ai_reply(message, model=model, system_message=system_message, message_history=history_state)\n",
+    "    except Exception as e:\n",
+    "        # If an error occurs, raise a Gradio error\n",
+    "        raise gr.Error(e)\n",
+    "    \n",
+    "    # Append the user's message and the AI's reply to the chatbot_messages list\n",
+    "    chatbot_messages.append((message, ai_reply))\n",
+    "    \n",
+    "    # Append the user's message and the AI's reply to the history_state list\n",
+    "    history_state.append({\"role\": \"user\", \"content\": message})\n",
+    "    history_state.append({\"role\": \"assistant\", \"content\": ai_reply})\n",
+    "    \n",
+    "    # Return None (empty out the user's message textbox), the updated chatbot_messages, and the updated history_state\n",
+    "    return None, chatbot_messages, history_state"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "44d9fde1",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
+   "source": [
+    "The following snippet adjusts the Gradio interface to include examples (included in a separate file in this repo), model selection, prompts or 'system' messages, storing conversation history."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 28,
+   "id": "d45439f3",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
+   "outputs": [],
+   "source": [
+    "import examples as chatbot_examples\n",
+    "\n",
+    "# Define a function to return a chatbot app using Gradio\n",
+    "def get_chatbot_app(additional_examples=[], share=False):\n",
+    "    # Load chatbot examples and merge with any additional examples provided\n",
+    "    examples = chatbot_examples.load_examples(additional=additional_examples)\n",
+    "    \n",
+    "    # Define a function to get the names of the examples\n",
+    "    def get_examples():\n",
+    "        return [example[\"name\"] for example in examples]\n",
+    "\n",
+    "    # Define a function to choose an example based on the index\n",
+    "    def choose_example(index):\n",
+    "        system_message = examples[index][\"system_message\"].strip()\n",
+    "        user_message = examples[index][\"message\"].strip()\n",
+    "        return system_message, user_message, [], []\n",
+    "\n",
+    "    # Create the Gradio interface using the Blocks layout\n",
+    "    with gr.Blocks() as app:\n",
+    "        with gr.Tab(\"Conversation\"):\n",
+    "            with gr.Row():\n",
+    "                with gr.Column():\n",
+    "                    # Create a dropdown to select examples\n",
+    "                    example_dropdown = gr.Dropdown(get_examples(), label=\"Examples\", type=\"index\")\n",
+    "                    # Create a button to load the selected example\n",
+    "                    example_load_btn = gr.Button(value=\"Load\")\n",
+    "                    # Create a textbox for the system message (prompt)\n",
+    "                    system_message = gr.Textbox(label=\"System Message (Prompt)\", value=\"You are a helpful assistant.\")\n",
+    "                with gr.Column():\n",
+    "                    # Create a dropdown to select the AI model\n",
+    "                    model_selector = gr.Dropdown(\n",
+    "                        [\"gpt-3.5-turbo\", \"gpt-4\"],\n",
+    "                        label=\"Model\",\n",
+    "                        value=\"gpt-3.5-turbo\"\n",
+    "                    )\n",
+    "                    # Create a chatbot interface for the conversation\n",
+    "                    chatbot = gr.Chatbot(label=\"Conversation\")\n",
+    "                    # Create a textbox for the user's message\n",
+    "                    message = gr.Textbox(label=\"Message\")\n",
+    "                    # Create a state object to store the conversation history\n",
+    "                    history_state = gr.State()\n",
+    "                    # Create a button to send the user's message\n",
+    "                    btn = gr.Button(value=\"Send\")\n",
+    "\n",
+    "                # Connect the example load button to the choose_example function\n",
+    "                example_load_btn.click(choose_example, inputs=[example_dropdown], outputs=[system_message, message, chatbot, history_state])\n",
+    "                # Connect the send button to the chat function\n",
+    "                btn.click(chat, inputs=[model_selector, system_message, message, chatbot, history_state], outputs=[message, chatbot, history_state])\n",
+    "        # Return the app\n",
+    "        return app"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 29,
+   "id": "e166c472",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Running on local URL:  http://127.0.0.1:7881\n",
+      "\n",
+      "To create a public link, set `share=True` in `launch()`.\n"
+     ]
+    },
+    {
+     "data": {
+      "text/html": [
+       "<div><iframe src=\"http://127.0.0.1:7881/\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/plain": []
+     },
+     "execution_count": 29,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "# Call the launch_chatbot function to start the chatbot interface using Gradio\n",
+    "# Set the share parameter to False, meaning the interface will not be publicly accessible\n",
+    "get_chatbot_app().launch()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "ae24e591",
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "celltoolbar": "Raw Cell Format",
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.8"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

02_deploying_a_chatbot.ipynb → 05_deploying_a_chatbot_to_the_web.ipynb

@@ -9,74 +9,12 @@
     }
    },
    "source": [
-    "# Lecture 2: Deploying a basic Chatbot using OpenAI\n",
+    "# Deploying a Chatbot to the Web\n",
     "* **Created by:** Eric Martinez\n",
     "* **For:** Software Engineering 2\n",
     "* **At:** University of Texas Rio-Grande Valley"
    ]
   },
-  {
-   "cell_type": "markdown",
-   "id": "e989bbe3",
-   "metadata": {
-    "slideshow": {
-     "slide_type": "slide"
-    }
-   },
-   "source": [
-    "## Tools and Concepts"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "368cda1a",
-   "metadata": {
-    "slideshow": {
-     "slide_type": "slide"
-    }
-   },
-   "source": [
-    "#### Gradio\n",
-    "\n",
-    "Gradio is an open-source Python library that allows developers to create and prototype user interfaces (UIs) and APIs for machine learning applications and LLMs quickly and easily. Three reasons it is a great tool are:\n",
-    "\n",
-    "* Simple and intuitive: Gradio makes it easy to create UIs with minimal code, allowing developers to focus on their models.\n",
-    "* Versatile: Gradio supports a wide range of input and output types, making it suitable for various machine learning tasks.\n",
-    "* Shareable: Gradio interfaces can be shared with others, enabling collaboration and easy access to your models."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "e2888a24",
-   "metadata": {
-    "slideshow": {
-     "slide_type": "slide"
-    }
-   },
-   "source": [
-    "#### Chatbots\n",
-    "\n",
-    "Chatbots are AI-powered conversational agents designed to interact with users through natural language. LLMs have the potential to revolutionize chatbots by providing more human-like, accurate, and context-aware responses, leading to more engaging and useful interactions."
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "6c3f79e3",
-   "metadata": {
-    "slideshow": {
-     "slide_type": "slide"
-    }
-   },
-   "source": [
-    "#### OpenAI API\n",
-    "\n",
-    "The OpenAI API provides access to powerful LLMs like GPT-3.5 and GPT-4, enabling developers to leverage these models in their applications. To access the API, sign up for an API key on the OpenAI website and follow the documentation to make API calls.\n",
-    "\n",
-    "For enterprise: Azure OpenAI offers a robust and scalable platform for deploying LLMs in enterprise applications. It provides features like security, compliance, and support, making it an ideal choice for businesses looking to leverage LLMs.\n",
-    "\n",
-    "[Sign-up for OpenAI API Access](https://platform.openai.com/signup)"
-   ]
-  },
   {
    "cell_type": "markdown",
    "id": "ffb051ff",
@@ -86,673 +24,85 @@
     }
    },
    "source": [
-    "#### HuggingFace\n",
+    "## HuggingFace\n",
     "\n",
     "HuggingFace is an AI research organization and platform that provides access to a wide range of pre-trained LLMs and tools for training, fine-tuning, and deploying models. It has a user-friendly interface and a large community, making it a popular choice for working with LLMs."
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "6f24bac1",
-   "metadata": {
-    "slideshow": {
-     "slide_type": "slide"
-    }
-   },
-   "source": [
-    "#### Example: Gradio Interface\n"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "4ecdb50a",
-   "metadata": {
-    "slideshow": {
-     "slide_type": "fragment"
-    }
-   },
-   "outputs": [],
-   "source": [
-    "!pip -q install --upgrade gradio"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "0b28a4e7",
-   "metadata": {
-    "slideshow": {
-     "slide_type": "fragment"
-    }
-   },
-   "outputs": [],
-   "source": [
-    "import gradio as gr\n",
-    " \n",
-    "def do_something_cool(first_name, last_name):\n",
-    "    return f\"{first_name} {last_name} is so cool\"\n",
-    "    \n",
-    "with gr.Blocks() as demo:\n",
-    "        first_name_box = gr.Textbox(label=\"First Name\")\n",
-    "        last_name_box = gr.Textbox(label=\"Last Name\")\n",
-    "        output_box = gr.Textbox(label=\"Output\", interactive=False)\n",
-    "        btn = gr.Button(value =\"Send\")\n",
-    "        btn.click(do_something_cool, inputs = [first_name_box, last_name_box], outputs = [output_box])\n",
-    "    demo.launch(share=True)\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "c1b1b69e",
+   "id": "8b3aec8b",
    "metadata": {
     "slideshow": {
      "slide_type": "slide"
     }
    },
    "source": [
-    "#### Example: Basic Gradio Chatbot Interface / Just the look"
+    "## Deploying to HuggingFace"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "ea2878bb",
-   "metadata": {
-    "slideshow": {
-     "slide_type": "fragment"
-    }
-   },
-   "source": [
-    "Version with minimal comments"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "a6f1cca3",
-   "metadata": {
-    "slideshow": {
-     "slide_type": "fragment"
-    }
-   },
-   "outputs": [],
+   "id": "7804a8ce",
+   "metadata": {},
    "source": [
-    "import gradio as gr\n",
-    "\n",
-    "def chat(message, history):\n",
-    "    history = history or []\n",
-    "    # Set a simple AI reply (replace this with a call to an LLM for a more sophisticated response)\n",
-    "    ai_reply = \"hello\"      \n",
-    "    history.append((message, ai_reply))\n",
-    "    return None, history, history\n",
-    "    \n",
-    "with gr.Blocks() as demo:\n",
-    "    with gr.Tab(\"Conversation\"):\n",
-    "        with gr.Row():\n",
-    "            with gr.Column():\n",
-    "                chatbot = gr.Chatbot(label=\"Conversation\")\n",
-    "                message = gr.Textbox(label=\"Message\")\n",
-    "                history_state = gr.State()\n",
-    "                btn = gr.Button(value =\"Send\")\n",
-    "            btn.click(chat, inputs = [message, history_state], outputs = [message, chatbot, history_state])\n",
-    "    demo.launch(share=True)\n"
+    "#### Configuring the files required"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "5cc72efb",
-   "metadata": {
-    "slideshow": {
-     "slide_type": "slide"
-    }
-   },
-   "source": [
-    "Version with more comments"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "44debca6",
-   "metadata": {
-    "slideshow": {
-     "slide_type": "fragment"
-    }
-   },
-   "outputs": [],
+   "id": "60c8e7f6",
+   "metadata": {},
    "source": [
-    "import gradio as gr\n",
-    "\n",
-    "# Define the chat function that processes user input and generates an AI response\n",
-    "def chat(message, history):\n",
-    "    # If history is empty, initialize it as an empty list\n",
-    "    history = history or []\n",
-    "    # Set a simple AI reply (replace this with a call to an LLM for a more sophisticated response)\n",
-    "    ai_reply = \"hello\"\n",
-    "    # Append the user message and AI reply to the conversation history\n",
-    "    history.append((message, ai_reply))\n",
-    "    # Return the updated history and display it in the chatbot interface\n",
-    "    return None, history, history\n",
-    "\n",
-    "# Create a Gradio Blocks interface\n",
-    "with gr.Blocks() as demo:\n",
-    "    # Create a tab for the conversation\n",
-    "    with gr.Tab(\"Conversation\"):\n",
-    "        # Create a row for the input components\n",
-    "        with gr.Row():\n",
-    "            # Create a column for the input components\n",
-    "            with gr.Column():\n",
-    "                # Create a chatbot component to display the conversation\n",
-    "                chatbot = gr.Chatbot(label=\"Conversation\")\n",
-    "                # Create a textbox for user input\n",
-    "                message = gr.Textbox(label=\"Message\")\n",
-    "                # Create a state variable to store the conversation history\n",
-    "                history_state = gr.State()\n",
-    "                # Create a button to send the user's message\n",
-    "                btn = gr.Button(value=\"Send\")\n",
-    "            # Connect the button click event to the chat function, passing in the input components and updating the output components\n",
-    "            btn.click(chat, inputs=[message, history_state], outputs=[message, chatbot, history_state])\n",
-    "    # Launch the Gradio interface and make it shareable\n",
-    "    demo.launch(share=True)"
+    "Let's face it! Once we start building cool stuff we are going to want to show it off. It can take us < 10 minutes to deploy our chatbots and LLM applications when using Gradio!"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "c412a4e9",
-   "metadata": {
-    "slideshow": {
-     "slide_type": "slide"
-    }
-   },
-   "source": [
-    "#### Example: Using the OpenAI API"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "bcc79375",
-   "metadata": {
-    "slideshow": {
-     "slide_type": "fragment"
-    }
-   },
-   "outputs": [],
-   "source": [
-    "!pip -q install --upgrade openai"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "c0183045",
-   "metadata": {
-    "slideshow": {
-     "slide_type": "fragment"
-    }
-   },
-   "outputs": [],
-   "source": [
-    "## Set your OpenAI API Key, or alternatively uncomment these lines and set it manually\n",
-    "# import os\n",
-    "# os.environ[\"OPENAI_API_KEY\"] = \"\""
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "af4895c3",
-   "metadata": {
-    "slideshow": {
-     "slide_type": "fragment"
-    }
-   },
-   "outputs": [],
+   "id": "54de0ddc",
+   "metadata": {},
    "source": [
-    "import openai\n",
-    "\n",
-    "# Uses OpenAI ChatCompletion to reply to the user's message\n",
-    "def get_ai_reply(message):\n",
-    "    completion = openai.ChatCompletion.create(model=\"gpt-3.5-turbo\", messages=[{\"role\": \"user\", \"content\": message}])\n",
-    "    return completion.choices[0].message.content"
+    "Add a username and password for your app to your `.env` file. This will ensure that unauthorized users are not able to access LLM features. Use the following format:"
    ]
   },
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "42a36d3a",
-   "metadata": {
-    "slideshow": {
-     "slide_type": "fragment"
-    }
-   },
-   "outputs": [],
-   "source": [
-    "get_ai_reply(\"hello\")"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "cd292e78",
-   "metadata": {
-    "slideshow": {
-     "slide_type": "slide"
-    }
-   },
-   "source": [
-    "## Tutorial: A Basic Conversational Chatbot with LLM (has limitations)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "dce44841",
-   "metadata": {
-    "slideshow": {
-     "slide_type": "fragment"
-    }
-   },
-   "source": [
-    "#### Installing Dependencies"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 8,
-   "id": "1bae55e3",
-   "metadata": {
-    "slideshow": {
-     "slide_type": "fragment"
-    }
-   },
-   "outputs": [],
-   "source": [
-    "!pip -q install --upgrade gradio\n",
-    "!pip -q install --upgrade openai"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "76d2c402",
-   "metadata": {
-    "slideshow": {
-     "slide_type": "fragment"
-    }
-   },
-   "source": [
-    "#### Setting OpenAI API Key"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 9,
-   "id": "5a006af0",
-   "metadata": {
-    "slideshow": {
-     "slide_type": "fragment"
-    }
-   },
-   "outputs": [],
-   "source": [
-    "## Set your OpenAI API Key, or alternatively uncomment these lines and set it manually\n",
-    "# import os\n",
-    "# os.environ[\"OPENAI_API_KEY\"] = \"\""
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "f762bbca",
-   "metadata": {
-    "slideshow": {
-     "slide_type": "slide"
-    }
-   },
-   "source": [
-    "#### Creating a basic Chatbot UI using Gradio"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 10,
-   "id": "feb92318",
-   "metadata": {
-    "slideshow": {
-     "slide_type": "fragment"
-    }
-   },
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "\n",
-      "Thanks for being a Gradio user! If you have questions or feedback, please join our Discord server and chat with us: https://discord.gg/feTf9x3ZSB\n",
-      "Running on local URL:  http://127.0.0.1:7862\n",
-      "Running on public URL: https://d3baeb9982f6708da1.gradio.live\n",
-      "\n",
-      "This share link expires in 72 hours. For free permanent hosting and GPU upgrades (NEW!), check out Spaces: https://huggingface.co/spaces\n"
-     ]
-    },
-    {
-     "data": {
-      "text/html": [
-       "<div><iframe src=\"https://d3baeb9982f6708da1.gradio.live\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
-      ],
-      "text/plain": [
-       "<IPython.core.display.HTML object>"
-      ]
-     },
-     "metadata": {},
-     "output_type": "display_data"
-    }
-   ],
-   "source": [
-    "import gradio as gr\n",
-    "import openai\n",
-    "\n",
-    "# Uses OpenAI ChatCompletion to reply to the user's message\n",
-    "def get_ai_reply(message):\n",
-    "    completion = openai.ChatCompletion.create(model=\"gpt-3.5-turbo\", messages=[{\"role\": \"user\", \"content\": message}])\n",
-    "    return completion.choices[0].message.content\n",
-    "\n",
-    "def chat(message, history):\n",
-    "    history = history or []\n",
-    "    ai_reply = get_ai_reply(message)       \n",
-    "    history.append((message, ai_reply))\n",
-    "    return None, history, history\n",
-    "    \n",
-    "with gr.Blocks() as demo:\n",
-    "    with gr.Tab(\"Conversation\"):\n",
-    "        with gr.Row():\n",
-    "            with gr.Column():\n",
-    "                chatbot = gr.Chatbot(label=\"Conversation\")\n",
-    "                message = gr.Textbox(label=\"Message\")\n",
-    "                history_state = gr.State()\n",
-    "                btn = gr.Button(value =\"Send\")\n",
-    "            btn.click(chat, inputs = [message, history_state], outputs = [message, chatbot, history_state])\n",
-    "    demo.launch(share=True)\n"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "77c9ec20",
-   "metadata": {
-    "slideshow": {
-     "slide_type": "slide"
-    }
-   },
-   "source": [
-    "#### Limitations\n",
-    "* Hardcoded to 'gpt-3.5-turbo'\n",
-    "* No error-handling on the API request\n",
-    "* Doesn't have memory\n",
-    "* Doesn't have a prompt or 'system' message"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "66215904",
-   "metadata": {
-    "slideshow": {
-     "slide_type": "slide"
-    }
-   },
-   "source": [
-    "## Tutorial: Improved Chatbot"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "b93b8dc8",
-   "metadata": {
-    "slideshow": {
-     "slide_type": "slide"
-    }
-   },
-   "source": [
-    "The following snippet adds the ability to adjust the prompt or 'system message, error-handing, and the conversation history to the API call."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 11,
-   "id": "89c48f97",
-   "metadata": {
-    "slideshow": {
-     "slide_type": "fragment"
-    }
-   },
-   "outputs": [],
-   "source": [
-    "import gradio as gr\n",
-    "import openai\n",
-    "import examples as chatbot_examples\n",
-    "\n",
-    "# Define a function to get the AI's reply using the OpenAI API\n",
-    "def get_ai_reply(model, system_message, message, history_state):\n",
-    "    # Initialize the messages list with the system message\n",
-    "    messages = [{\"role\": \"system\", \"content\": system_message}]\n",
-    "    \n",
-    "    # Add the conversation history to the messages list\n",
-    "    messages += history_state\n",
-    "    \n",
-    "    # Add the user's message to the messages list\n",
-    "    messages += [{\"role\": \"user\", \"content\": message}]\n",
-    "    \n",
-    "    # Make an API call to the OpenAI ChatCompletion endpoint with the model and messages\n",
-    "    completion = openai.ChatCompletion.create(\n",
-    "        model=model,\n",
-    "        messages=messages\n",
-    "    )\n",
-    "    \n",
-    "    # Extract and return the AI's response from the API response\n",
-    "    return completion.choices[0].message.content"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "99f57faf",
-   "metadata": {
-    "slideshow": {
-     "slide_type": "slide"
-    }
-   },
-   "source": [
-    "The following snippet adds conversation history to the Gradio chat functionality, handles erorors, and passes along the system message."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 12,
-   "id": "9e55e844",
-   "metadata": {
-    "slideshow": {
-     "slide_type": "fragment"
-    }
-   },
-   "outputs": [],
-   "source": [
-    "# Define a function to handle the chat interaction with the AI model\n",
-    "def chat(model, system_message, message, chatbot_messages, history_state):\n",
-    "    # Initialize chatbot_messages and history_state if they are not provided\n",
-    "    chatbot_messages = chatbot_messages or []\n",
-    "    history_state = history_state or []\n",
-    "    \n",
-    "    # Try to get the AI's reply using the get_ai_reply function\n",
-    "    try:\n",
-    "        ai_reply = get_ai_reply(model, system_message, message, history_state)\n",
-    "    except:\n",
-    "        # If an error occurs, return None and the current chatbot_messages and history_state\n",
-    "        return None, chatbot_messages, history_state\n",
-    "    \n",
-    "    # Append the user's message and the AI's reply to the chatbot_messages list\n",
-    "    chatbot_messages.append((message, ai_reply))\n",
-    "    \n",
-    "    # Append the user's message and the AI's reply to the history_state list\n",
-    "    history_state.append({\"role\": \"user\", \"content\": message})\n",
-    "    history_state.append({\"role\": \"assistant\", \"content\": ai_reply})\n",
-    "    \n",
-    "    # Return None (empty out the user's message textbox), the updated chatbot_messages, and the updated history_state\n",
-    "    return None, chatbot_messages, history_state"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "44d9fde1",
-   "metadata": {
-    "slideshow": {
-     "slide_type": "slide"
-    }
-   },
-   "source": [
-    "The following snippet adjusts the Gradio interface to include examples (included in a separate file in this repo), model selection, prompts or 'system' messages, storing conversation history."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 13,
-   "id": "d45439f3",
-   "metadata": {
-    "slideshow": {
-     "slide_type": "fragment"
-    }
-   },
-   "outputs": [],
-   "source": [
-    "# Define a function to launch the chatbot interface using Gradio\n",
-    "def launch_chatbot(additional_examples=[], share=False):\n",
-    "    # Load chatbot examples and merge with any additional examples provided\n",
-    "    examples = chatbot_examples.load_examples(additional=additional_examples)\n",
-    "    \n",
-    "    # Define a function to get the names of the examples\n",
-    "    def get_examples():\n",
-    "        return [example[\"name\"] for example in examples]\n",
-    "\n",
-    "    # Define a function to choose an example based on the index\n",
-    "    def choose_example(index):\n",
-    "        system_message = examples[index][\"system_message\"].strip()\n",
-    "        user_message = examples[index][\"message\"].strip()\n",
-    "        return system_message, user_message, [], []\n",
-    "\n",
-    "    # Create the Gradio interface using the Blocks layout\n",
-    "    with gr.Blocks() as demo:\n",
-    "        with gr.Tab(\"Conversation\"):\n",
-    "            with gr.Row():\n",
-    "                with gr.Column():\n",
-    "                    # Create a dropdown to select examples\n",
-    "                    example_dropdown = gr.Dropdown(get_examples(), label=\"Examples\", type=\"index\")\n",
-    "                    # Create a button to load the selected example\n",
-    "                    example_load_btn = gr.Button(value=\"Load\")\n",
-    "                    # Create a textbox for the system message (prompt)\n",
-    "                    system_message = gr.Textbox(label=\"System Message (Prompt)\", value=\"You are a helpful assistant.\")\n",
-    "                with gr.Column():\n",
-    "                    # Create a dropdown to select the AI model\n",
-    "                    model_selector = gr.Dropdown(\n",
-    "                        [\"gpt-3.5-turbo\", \"gpt-4\"],\n",
-    "                        label=\"Model\",\n",
-    "                        value=\"gpt-3.5-turbo\"\n",
-    "                    )\n",
-    "                    # Create a chatbot interface for the conversation\n",
-    "                    chatbot = gr.Chatbot(label=\"Conversation\")\n",
-    "                    # Create a textbox for the user's message\n",
-    "                    message = gr.Textbox(label=\"Message\")\n",
-    "                    # Create a state object to store the conversation history\n",
-    "                    history_state = gr.State()\n",
-    "                    # Create a button to send the user's message\n",
-    "                    btn = gr.Button(value=\"Send\")\n",
-    "\n",
-    "                # Connect the example load button to the choose_example function\n",
-    "                example_load_btn.click(choose_example, inputs=[example_dropdown], outputs=[system_message, message, chatbot, history_state])\n",
-    "                # Connect the send button to the chat function\n",
-    "                btn.click(chat, inputs=[model_selector, system_message, message, chatbot, history_state], outputs=[message, chatbot, history_state])\n",
-    "        # Launch the Gradio interface\n",
-    "        demo.launch(share=share)"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": 7,
-   "id": "e166c472",
-   "metadata": {
-    "slideshow": {
-     "slide_type": "slide"
-    }
-   },
-   "outputs": [
-    {
-     "name": "stdout",
-     "output_type": "stream",
-     "text": [
-      "Running on local URL:  http://127.0.0.1:7861\n",
-      "\n",
-      "To create a public link, set `share=True` in `launch()`.\n"
-     ]
-    },
-    {
-     "data": {
-      "text/html": [
-       "<div><iframe src=\"http://127.0.0.1:7861/\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
-      ],
-      "text/plain": [
-       "<IPython.core.display.HTML object>"
-      ]
-     },
-     "metadata": {},
-     "output_type": "display_data"
-    }
-   ],
-   "source": [
-    "# Call the launch_chatbot function to start the chatbot interface using Gradio\n",
-    "# Set the share parameter to False, meaning the interface will not be publicly accessible\n",
-    "launch_chatbot(share=False)"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "8b3aec8b",
-   "metadata": {
-    "slideshow": {
-     "slide_type": "slide"
-    }
-   },
-   "source": [
-    "## Deploying to HuggingFace"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "c3c33028",
+   "id": "95dec7cf",
    "metadata": {},
+   "outputs": [],
    "source": [
-    "#### Configuring the files required"
+    "APP_USERNAME=<whatever username you want>\n",
+    "APP_PASSWORD=<whatever password you want>"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "32e5fa47",
+   "id": "5072dc21",
    "metadata": {},
    "source": [
-    "Let's face it! Once we start building cool stuff we are going to want to show it off. It can take us < 10 minutes to deploy our chatbots and LLM applications when using Gradio!"
+    "Let's start by taking all of our necessary chatbot code into one file which we will name `app.py`. Run the following cell to automatically write it!"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "a0b90836",
+   "id": "aacdcaaf",
    "metadata": {},
    "source": [
-    "Let's start by taking all of our necessary chatbot code into one file which we will name `app.py`. Run the following cell to automatically write it!"
+    "Take note that this code has been altered a little bit from the last chatbot example in order to add authentication."
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 2,
-   "id": "44f7664b",
+   "execution_count": 10,
+   "id": "710b66f7",
    "metadata": {},
    "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Writing app.py\n"
+      "Overwriting app.py\n"
      ]
     }
    ],
@@ -762,16 +112,37 @@
     "import openai\n",
     "import examples as chatbot_examples\n",
     "from dotenv import load_dotenv\n",
+    "import os\n",
     "\n",
-    "load_dotenv()\n",
+    "load_dotenv()  # take environment variables from .env.\n",
     "\n",
+    "# In order to authenticate, secrets must have been set, and the user supplied credentials match\n",
+    "def auth(username, password):\n",
+    "    app_username = os.getenv(\"APP_USERNAME\")\n",
+    "    app_password = os.getenv(\"APP_PASSWORD\")\n",
+    "\n",
+    "    if app_username and app_password:\n",
+    "        if(username == app_username and password == app_password):\n",
+    "            print(\"Logged in successfully.\")\n",
+    "            return True\n",
+    "        else:\n",
+    "            print(\"Username or password does not match.\")\n",
+    "    else:\n",
+    "        print(\"Credential secrets not set.\")\n",
+    "    return False\n",
+    "    \n",
     "# Define a function to get the AI's reply using the OpenAI API\n",
-    "def get_ai_reply(model, system_message, message, history_state):\n",
-    "    # Initialize the messages list with the system message\n",
-    "    messages = [{\"role\": \"system\", \"content\": system_message}]\n",
+    "def get_ai_reply(message, model=\"gpt-3.5-turbo\", system_message=None, temperature=0, message_history=[]):\n",
+    "    # Initialize the messages list\n",
+    "    messages = []\n",
     "    \n",
-    "    # Add the conversation history to the messages list\n",
-    "    messages += history_state\n",
+    "    # Add the system message to the messages list\n",
+    "    if system_message is not None:\n",
+    "        messages += [{\"role\": \"system\", \"content\": system_message}]\n",
+    "\n",
+    "    # Add the message history to the messages list\n",
+    "    if message_history is not None:\n",
+    "        messages += message_history\n",
     "    \n",
     "    # Add the user's message to the messages list\n",
     "    messages += [{\"role\": \"user\", \"content\": message}]\n",
@@ -779,11 +150,12 @@
     "    # Make an API call to the OpenAI ChatCompletion endpoint with the model and messages\n",
     "    completion = openai.ChatCompletion.create(\n",
     "        model=model,\n",
-    "        messages=messages\n",
+    "        messages=messages,\n",
+    "        temperature=temperature\n",
     "    )\n",
     "    \n",
     "    # Extract and return the AI's response from the API response\n",
-    "    return completion.choices[0].message.content\n",
+    "    return completion.choices[0].message.content.strip()\n",
     "\n",
     "# Define a function to handle the chat interaction with the AI model\n",
     "def chat(model, system_message, message, chatbot_messages, history_state):\n",
@@ -793,10 +165,10 @@
     "    \n",
     "    # Try to get the AI's reply using the get_ai_reply function\n",
     "    try:\n",
-    "        ai_reply = get_ai_reply(model, system_message, message, history_state)\n",
-    "    except:\n",
-    "        # If an error occurs, return None and the current chatbot_messages and history_state\n",
-    "        return None, chatbot_messages, history_state\n",
+    "        ai_reply = get_ai_reply(message, model=model, system_message=system_message, message_history=history_state)\n",
+    "    except Exception as e:\n",
+    "        # If an error occurs, raise a Gradio error\n",
+    "        raise gr.Error(e)\n",
     "    \n",
     "    # Append the user's message and the AI's reply to the chatbot_messages list\n",
     "    chatbot_messages.append((message, ai_reply))\n",
@@ -809,7 +181,7 @@
     "    return None, chatbot_messages, history_state\n",
     "\n",
     "# Define a function to launch the chatbot interface using Gradio\n",
-    "def launch_chatbot(additional_examples=[], share=False):\n",
+    "def get_chatbot_app(additional_examples=[]):\n",
     "    # Load chatbot examples and merge with any additional examples provided\n",
     "    examples = chatbot_examples.load_examples(additional=additional_examples)\n",
     "    \n",
@@ -819,12 +191,15 @@
     "\n",
     "    # Define a function to choose an example based on the index\n",
     "    def choose_example(index):\n",
-    "        system_message = examples[index][\"system_message\"].strip()\n",
-    "        user_message = examples[index][\"message\"].strip()\n",
-    "        return system_message, user_message, [], []\n",
+    "        if(index!=None):\n",
+    "            system_message = examples[index][\"system_message\"].strip()\n",
+    "            user_message = examples[index][\"message\"].strip()\n",
+    "            return system_message, user_message, [], []\n",
+    "        else:\n",
+    "            return \"\", \"\", [], []\n",
     "\n",
     "    # Create the Gradio interface using the Blocks layout\n",
-    "    with gr.Blocks() as demo:\n",
+    "    with gr.Blocks() as app:\n",
     "        with gr.Tab(\"Conversation\"):\n",
     "            with gr.Row():\n",
     "                with gr.Column():\n",
@@ -837,7 +212,7 @@
     "                with gr.Column():\n",
     "                    # Create a dropdown to select the AI model\n",
     "                    model_selector = gr.Dropdown(\n",
-    "                        [\"gpt-3.5-turbo\", \"gpt-4\"],\n",
+    "                        [\"gpt-3.5-turbo\"],\n",
     "                        label=\"Model\",\n",
     "                        value=\"gpt-3.5-turbo\"\n",
     "                    )\n",
@@ -854,17 +229,20 @@
     "                example_load_btn.click(choose_example, inputs=[example_dropdown], outputs=[system_message, message, chatbot, history_state])\n",
     "                # Connect the send button to the chat function\n",
     "                btn.click(chat, inputs=[model_selector, system_message, message, chatbot, history_state], outputs=[message, chatbot, history_state])\n",
-    "        # Launch the Gradio interface\n",
-    "        demo.launch(share=share)\n",
+    "        # Return the app\n",
+    "        return app\n",
     "        \n",
     "# Call the launch_chatbot function to start the chatbot interface using Gradio\n",
     "# Set the share parameter to False, meaning the interface will not be publicly accessible\n",
-    "launch_chatbot(share=False)"
+    "app = get_chatbot_app((\n",
+    "app.queue()  # <-- Sets up a queue with default parameters\n",
+    "    \n",
+    "launch(auth=auth)"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "2b1e34a5",
+   "id": "6d75af66",
    "metadata": {},
    "source": [
     "We will also need a `requirements.txt` file to store the list of the packages that HuggingFace needs to install to run our chatbot."
@@ -872,27 +250,28 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 5,
-   "id": "cd40b970",
+   "execution_count": 11,
+   "id": "14d0e434",
    "metadata": {},
    "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Writing requirements.txt\n"
+      "Overwriting requirements.txt\n"
      ]
     }
    ],
    "source": [
     "%%writefile requirements.txt\n",
-    "gradio\n",
-    "openai"
+    "gradio == 3.27.0\n",
+    "openai == 0.27.4\n",
+    "python-dotenv == 1.0.0"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "fd712cef",
+   "id": "4debec45",
    "metadata": {},
    "source": [
     "Now let's go ahead and commit our changes"
@@ -901,7 +280,7 @@
   {
    "cell_type": "code",
    "execution_count": 9,
-   "id": "36511322",
+   "id": "14d42a96",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -910,8 +289,8 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 10,
-   "id": "ad525ca6",
+   "execution_count": 8,
+   "id": "d7c5b127",
    "metadata": {},
    "outputs": [],
    "source": [
@@ -921,7 +300,7 @@
   {
    "cell_type": "code",
    "execution_count": 11,
-   "id": "6d52b2bf",
+   "id": "18960d9f",
    "metadata": {},
    "outputs": [
     {
@@ -941,112 +320,121 @@
   },
   {
    "cell_type": "markdown",
-   "id": "d0578c17",
+   "id": "09221ee0",
    "metadata": {},
    "source": [
-    "#### Protecting our secrets with .env"
+    "#### Using HuggingFace Spaces"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "e6626d53",
+   "id": "db789c94",
    "metadata": {},
    "source": [
-    "Certain things like encryption keys, database connection strings, usernames, passwords, API keys, etc should NEVER be stored in code or version control.\n",
-    "\n",
-    "We have two secrets:\n",
-    "* OpenAI API Key\n",
-    "* HuggingFace token\n",
-    "\n",
-    "We want some way of accessing this information, but somehow not including it in version control.\n",
-    "\n",
-    "To do this we will use the library `python-dotenv` to store our secrets in a file `.env` but we will also be careful not to put them in version control.\n",
-    "\n",
-    "To do this you must add this file to the `.gitignore` file in this repository, which I have already done for you."
+    "As mentioned before, HuggingFace is a free-to-use platform for hosting AI demos and apps. We will need to make a HuggingFace _Space_ for our chatbot."
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "bfb813ae",
+   "id": "d9eedd10",
    "metadata": {},
    "source": [
-    "Create a `.env` file in this folder with the following contents:"
+    "First sign up for a free HuggingFace account [here](https://huggingface.co/join). After you sign up, create a new Space by clicking \"New Space\" on the navigation menu (press on your profile image)."
    ]
   },
   {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "67f06456",
+   "cell_type": "markdown",
+   "id": "3e042d24",
+   "metadata": {},
+   "source": [
+    "#### Generate a HuggingFace Access Token"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "a7d0781d",
    "metadata": {},
-   "outputs": [],
    "source": [
-    "OPENAI_API_KEY=<your api key>\n",
-    "HF_TOKEN=<your hugging face token>"
+    "#### Login to HuggingFace Hub"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "21db9e36",
+   "id": "eba83252",
    "metadata": {},
    "source": [
-    "Then, double check that your `.gitignore` contains this line:"
+    "Install `huggingface_hub`"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": null,
-   "id": "cd4ccd52",
+   "execution_count": 40,
+   "id": "266bf481",
    "metadata": {},
    "outputs": [],
    "source": [
-    ".env"
+    "!pip -q install --upgrade huggingface_hub"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "88118322",
+   "id": "d50cd84b",
    "metadata": {},
    "source": [
-    "#### Using HuggingFace Spaces"
+    "Login to HuggingFace"
    ]
   },
   {
-   "cell_type": "markdown",
-   "id": "d8c597ff",
+   "cell_type": "code",
+   "execution_count": 6,
+   "id": "53fd5037",
    "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Token is valid.\n",
+      "Your token has been saved in your configured git credential helpers (osxkeychain).\n",
+      "Your token has been saved to /Users/ericmartinez/.cache/huggingface/token\n",
+      "Login successful\n"
+     ]
+    }
+   ],
    "source": [
-    "As mentioned before, HuggingFace is a free-to-use platform for hosting AI demos and apps. We will need to make a HuggingFace _Space_ for our chatbot."
+    "from huggingface_hub import notebook_login\n",
+    "notebook_login()"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "173a5621",
+   "id": "90f9bd4d",
    "metadata": {},
    "source": [
-    "First sign up for a free HuggingFace account [here](https://huggingface.co/join). After you sign up, create a new Space by clicking \"New Space\" on the navigation menu (press on your profile image)."
+    "#### Now lets setup git and HuggingFace Spaces to work together and deploy"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "a81a99b8",
+   "id": "66468481",
    "metadata": {},
    "source": [
-    "Now lets setup git and HuggingFace Spaces to work together."
+    "<span style=\"color:red\">REPLACE MY URL WITH YOURS</span>"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 12,
-   "id": "e83e1ff1",
+   "execution_count": null,
+   "id": "827a201d",
    "metadata": {},
    "outputs": [],
    "source": [
-    "!git remote add huggingface <enter your hugging face url>"
+    "!git remote add huggingface https://huggingface.co/spaces/ericmichael/gradio-chatbot-demo"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "fd09c53b",
+   "id": "f8b3bb3d",
    "metadata": {},
    "source": [
     "Then force push to sync everything for the first time."
@@ -1054,20 +442,30 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
-   "id": "3c1c41a0",
+   "execution_count": 12,
+   "id": "86c9ee4e",
    "metadata": {},
    "outputs": [
     {
      "name": "stdout",
      "output_type": "stream",
      "text": [
-      "Username for 'https://huggingface.co': "
+      "Total 0 (delta 0), reused 0 (delta 0), pack-reused 0\n",
+      "To https://huggingface.co/spaces/ericmichael/gradio-chatbot-demo\n",
+      " + 8911ec0...3693bcc main -> main (forced update)\n"
      ]
     }
    ],
    "source": [
-    "!git push --force huggingface master"
+    "!git push --force huggingface main"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "3a353ebf",
+   "metadata": {},
+   "source": [
+    "That's it! 🎉 Check your HuggingFace Space URL to access your chatbot!"
    ]
   },
   {

03_designing_effective_prompts.ipynb → 06_designing_effective_prompts.ipynb

@@ -9,7 +9,7 @@
     }
    },
    "source": [
-    "# Lecture 3: Designing Effective Prompts for Practical LLM Applications\n",
+    "# Designing Effective Prompts for Practical LLM Applications\n",
     "* **Created by:** Eric Martinez\n",
     "* **For:** Software Engineering 2\n",
     "* **At:** University of Texas Rio-Grande Valley"

04_software_engineering_applied_to_llms.ipynb → 07_software_engineering_applied_to_llms.ipynb

@@ -9,12 +9,20 @@
     }
    },
    "source": [
-    "# Lecture 4: Software Engineering Applied to LLMs\n",
+    "# Software Engineering Applied to LLMs\n",
     "* **Created by:** Eric Martinez\n",
     "* **For:** Software Engineering 2\n",
     "* **At:** University of Texas Rio-Grande Valley"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "02d71e5e",
+   "metadata": {},
+   "source": [
+    "## Concerns with Quality and Performance Issues of LLM Integrated Apps"
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "60fef658",
@@ -24,14 +32,20 @@
     }
    },
    "source": [
-    "## Quality and Performance Issues\n",
-    "\n",
     "* Applications depend on external APIs which has issues with flakiness and pricing, how do we avoid hitting APIs in testing?\n",
     "* Responses may not be correct or accurate, how do we increase confidence in result?\n",
     "* Responses may be biased or unethical or unwanted output, how do we stop this type of output?\n",
     "* User requests could be unethical or unwanted input, how do we filter this type of input?\n"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "5ef5865b",
+   "metadata": {},
+   "source": [
+    "## Lessons from Software Engineering Applied to LLMS:"
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "2fc1b19a",
@@ -41,7 +55,7 @@
     }
    },
    "source": [
-    "## Prototyping\n",
+    "#### Prototyping\n",
     "* Develop prompt prototypes early when working with customers or stakeholders, it is fast and cheap to test that the idea will work.\n",
     "* Test against realistic examples, early. Fail fast and iterate quickly.\n",
     "* Make a plan for how you will source dynamic data. If there is no path, the project is dead in the water."
@@ -56,7 +70,7 @@
     }
    },
    "source": [
-    "## Testing\n",
+    "#### Testing\n",
     "* Unit test prompts using traditional methods to increase confidence.\n",
     "* Unit test your prompts using LLMs to increase confidence.\n",
     "* Write tests that handle API errors or bad output (malformed, incorrect, unethical).\n",
@@ -72,7 +86,7 @@
     }
    },
    "source": [
-    "## Handling Bad Output\n",
+    "#### Handling Bad Output\n",
     "* Develop 'retry' mechanisms when you get unwanted output.\n",
     "* Develop specific prompts for different 'retry' conditions. Include the context, what went wrong, and what needs to be fixed.\n",
     "* Consider adding logging to your app to keep track of how often your app gets bad output."
@@ -87,7 +101,7 @@
     }
    },
    "source": [
-    "## Template Languages and Version Control\n",
+    "#### Template Languages and Version Control\n",
     "* Consider writing your prompt templates in dynamic template languages like ERB, Handlebars, etc.\n",
     "* Keep prompt templates and prompts in version control in your app's repo.\n",
     "* Write tests for handling template engine errors."
@@ -102,7 +116,7 @@
     }
    },
    "source": [
-    "## Prompt Injection/Leakage\n",
+    "#### Prompt Injection/Leakage\n",
     "* User-facing prompts should be tested against prompt injection attacks\n",
     "* Validate input at the UI and LLM level\n",
     "* Consider using an LLM to check if an output is similar to the prompt\n",
@@ -118,24 +132,97 @@
     }
    },
    "source": [
-    "## Security\n",
+    "#### Security\n",
     "* **Do not:** store API keys in application code as strings, encrypted or not.\n",
     "* **Do not:** store API keys in compiled binaries distributed to users.\n",
     "* **Do not:** store API keys in metadeta files bundled with your application.\n",
-    "* **Do:** create an intermediate web app (or API) with authentication/authorization that delegates requests to LLMs at run-time for use in front-end applications\n",
-    "* **Do:** if your front-end application does not have user accounts, consider implementing guest or anonymous accounts and expiring or rotating keys\n",
-    "* **Do:** when allowing LLMs to use tools, consider designing systems to pass-through user ids to tools so that they tools operate at the same level of access as the end-user"
+    "* **Do:** store API keys in environment variables or cloud secrets.\n",
+    "* **Do:** store API keys in a `.env` file that is blocked from version control. (Ideally these are encrypted with a secret that is not in version control, but that is beyond the scope of today's discussion.)\n",
+    "* **Do:** create an intermediate web app (or API) with authentication/authorization that delegates requests to LLMs at run-time for use in front-end applications.\n",
+    "* **Do:** if your front-end application does not have user accounts, consider implementing guest or anonymous accounts and expiring or rotating keys.\n",
+    "* **Do:** when allowing LLMs to use tools, consider designing systems to pass-through user ids to tools so that they tools operate at the same level of access as the end-user."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "9d3ef132",
+   "metadata": {},
+   "source": [
+    "## Production Deployment Considerations"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "a480904f",
+   "metadata": {},
+   "source": [
+    "* Wrap LLM features as web service or API. Don't give out your OpenAI keys directly in distributed software.\n",
+    "    - For example: Django, Flask, FastAPI, Express.js, Sinatra, Ruby on Rails"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "e6274d71",
+   "metadata": {},
+   "source": [
+    "* Consider whether there are any regulations that might impact how you handle data, such as GDPR and HIPAA.\n",
+    "    - Regulation may require specific data handling and storage practices.\n",
+    "    - Cloud providers may offer compliance certifications and assessment tools.\n",
+    "    - On-prem deployments can provide more control of data storage and processing, but might require more resources (hardware, people, software) for management and maintenance\n",
+    "    - Cloud providers like Azure have great tools like Azure Defender for Cloud and Microsoft Purview for managing compliance"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "2f9b5cf9",
+   "metadata": {},
+   "source": [
+    "- Using Cloud Services vs On-Prem\n",
+    "    - Cloud services offer many advantages such as scalability, flexibilitiy, cost-effectiveness, and ease of management.\n",
+    "    - Easy to spin up resources and scale based on demand, without worrying about infrastructure or maintenance.\n",
+    "    - Wide range of tools: performance optimization, monitoring, security, reliability."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "17111661",
+   "metadata": {},
+   "source": [
+    "- Container-based Architecture\n",
+    "    - Containerization is a lightweight virtualization method that packages an application and its dependencies into a single, portable unit called a container.\n",
+    "    - Containers can run consistently across different environments, making it easier to develop, test, and deploy applications. \n",
+    "    - Containerization is useful when you need to ensure consistent behavior across various platforms, simplify deployment and scaling, and improve resource utilization.\n",
+    "    - Common tools for deploying container-based architecture are Docker and Kubernetes."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "56890eec",
+   "metadata": {},
+   "source": [
+    "- Serverless Architectures\n",
+    "    - Serverless architectures are a cloud computing model where the cloud provider manages the infrastructure and automatically allocates resources based on the application's needs.\n",
+    "    - Developers only need to focus on writing code, and the provider takes care of scaling, patching, and maintaining the underlying infrastructure. \n",
+    "    - Serverless architectures can be useful when you want to reduce operational overhead, build event-driven applications, and optimize costs by paying only for the resources you actually use.\n",
+    "    - Common tools to build serverless applications and APIs include Azure Functions, AWS Lambda, and Google Cloud Functions."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "b390fa49",
+   "metadata": {},
+   "source": [
+    "- HuggingFace\n",
+    "    - Platforms like HuggingFace provide an ecosystem for sharing, collaborating, and deploying AI models, including LLMs. \n",
+    "    - They offer pre-trained models, tools, and APIs that simplify the development and integration of AI-powered applications. \n",
+    "    - These platforms can be useful when you want to leverage existing models, collaborate with the AI community, and streamline the deployment process for your LLM-based applications."
    ]
   },
   {
    "cell_type": "code",
    "execution_count": null,
-   "id": "181dd4ad",
-   "metadata": {
-    "slideshow": {
-     "slide_type": "slide"
-    }
-   },
+   "id": "72115a67",
+   "metadata": {},
    "outputs": [],
    "source": []
   }

05_escaping_the_sandbox.ipynb → 08_escaping_the_sandbox.ipynb

@@ -9,7 +9,7 @@
     }
    },
    "source": [
-    "# Lecture 5: Escaping the Sandbox and Unleashing Unlimited Power\n",
+    "# Escaping the Sandbox and Unleashing Unlimited Power\n",
     "* **Created by:** Eric Martinez\n",
     "* **For:** Software Engineering 2\n",
     "* **At:** University of Texas Rio-Grande Valley"