update

02_prototyping_a_basic_chatbot_ui.ipynb

@@ -87,14 +87,37 @@
   },
   {
    "cell_type": "code",
-   "execution_count": null,
+   "execution_count": 3,
    "id": "0b28a4e7",
    "metadata": {
     "slideshow": {
      "slide_type": "fragment"
     }
    },
-   "outputs": [],
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Running on local URL:  http://127.0.0.1:7882\n",
+      "Running on public URL: https://d807b00726b10425a4.gradio.live\n",
+      "\n",
+      "This share link expires in 72 hours. For free permanent hosting and GPU upgrades (NEW!), check out Spaces: https://huggingface.co/spaces\n"
+     ]
+    },
+    {
+     "data": {
+      "text/html": [
+       "<div><iframe src=\"https://d807b00726b10425a4.gradio.live\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
    "source": [
     "import gradio as gr\n",
     " \n",
@@ -102,11 +125,11 @@
     "    return f\"{first_name} {last_name} is so cool\"\n",
     "    \n",
     "with gr.Blocks() as app:\n",
-    "        first_name_box = gr.Textbox(label=\"First Name\")\n",
-    "        last_name_box = gr.Textbox(label=\"Last Name\")\n",
-    "        output_box = gr.Textbox(label=\"Output\", interactive=False)\n",
-    "        btn = gr.Button(value =\"Send\")\n",
-    "        btn.click(do_something_cool, inputs = [first_name_box, last_name_box], outputs = [output_box])\n",
+    "    first_name_box = gr.Textbox(label=\"First Name\")\n",
+    "    last_name_box = gr.Textbox(label=\"Last Name\")\n",
+    "    output_box = gr.Textbox(label=\"Output\", interactive=False)\n",
+    "    btn = gr.Button(value =\"Send\")\n",
+    "    btn.click(do_something_cool, inputs = [first_name_box, last_name_box], outputs = [output_box])\n",
     "    app.launch(share=True)\n"
    ]
   },
@@ -227,14 +250,18 @@
   {
    "cell_type": "markdown",
    "id": "3b660170",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
    "source": [
     "Sweet! That wasn't too bad, not much code to make a cool little UI!"
    ]
   }
  ],
  "metadata": {
-  "celltoolbar": "Raw Cell Format",
+  "celltoolbar": "Slideshow",
   "kernelspec": {
    "display_name": "Python 3 (ipykernel)",
    "language": "python",

03_using_the_openai_api.ipynb

@@ -44,32 +44,56 @@
     }
    },
    "source": [
-    "## Managing Application Secrets"
+    "## Managing Application Secrets\n",
+    "\n",
+    "Secrets are sensitive information, such as API keys, passwords, or cryptographic keys, that must be protected to ensure the security and integrity of a system.\n",
+    "\n",
+    "In software development, secrets are often used to authenticate users, grant access to resources, or encrypt/decrypt data. Mismanaging or exposing secrets can lead to severe security breaches and data leaks."
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "66de4ac1",
-   "metadata": {},
+   "id": "2059552f",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
    "source": [
-    "Secrets are sensitive information, such as API keys, passwords, or cryptographic keys, that must be protected to ensure the security and integrity of a system.\n",
-    "\n",
-    "In software development, secrets are often used to authenticate users, grant access to resources, or encrypt/decrypt data. Mismanaging or exposing secrets can lead to severe security breaches and data leaks.\n",
-    "\n",
-    "Common examples of secrets\n",
+    "#### Common examples of secrets\n",
     "* API keys\n",
     "* Database credentials\n",
     "* SSH keys\n",
     "* OAuth access tokens\n",
-    "* Encryption/decryption keys\n",
-    "\n",
-    "Common mistakes when handling secrets\n",
+    "* Encryption/decryption keys"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "a1e650f8",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
+   "source": [
+    "#### Common mistakes when handling secrets\n",
     "* Storing secrets in plain text\n",
     "* Hardcoding secrets in source code\n",
     "* Sharing secrets through unsecured channels (e.g., email or messaging apps)\n",
     "* Using the same secret for multiple purposes\n",
-    "* Not rotating or updating secrets regularly\n",
-    "\n",
+    "* Not rotating or updating secrets regularly"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "66de4ac1",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "skip"
+    }
+   },
+   "source": [
     "How attackers might obtain secrets\n",
     "* Exploiting vulnerabilities in software or infrastructure\n",
     "* Intercepting unencrypted communications\n",
@@ -91,7 +115,7 @@
     "* Limit access to secrets on a need-to-know basis\n",
     "* Implement proper auditing and monitoring of secret usage\n",
     "\n",
-    " Cloud services and secret management\n",
+    "Cloud services and secret management\n",
     "* Many cloud providers offer secret management services (e.g., AWS Secrets Manager, Azure Key Vault, Google Cloud Secret Manager) that securely store, manage, and rotate secrets.\n",
     "* These services often provide access control, encryption, and auditing capabilities.\n",
     "* Integrating cloud secret management services with your application can help secure sensitive information and reduce the risk of exposure.\n",
@@ -106,7 +130,11 @@
   {
    "cell_type": "markdown",
    "id": "ef366b65",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
    "source": [
     "#### Using  `.dotenv` library to protect secrets in Python"
    ]
@@ -114,27 +142,23 @@
   {
    "cell_type": "markdown",
    "id": "dc39df10",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
    "source": [
     " `.dotenv` is a Python library that allows developers to load environment variables from a `.env` file. It helps keep secrets out of source code and makes it easier to manage and update them."
    ]
   },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "88252d32",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "      3. Add secrets as key-value pairs in the `.env` file\n",
-    "      4. Load secrets in your Python code using the `load_dotenv()` function\n",
-    "      5. Access secrets using `os.environ`"
-   ]
-  },
   {
    "cell_type": "markdown",
    "id": "ae0500ea",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
    "source": [
     "Install the `python-dotenv` library"
    ]
@@ -143,7 +167,11 @@
    "cell_type": "code",
    "execution_count": 45,
    "id": "1212333f",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
    "outputs": [],
    "source": [
     "!pip -q install --upgrade python-dotenv"
@@ -152,7 +180,11 @@
   {
    "cell_type": "markdown",
    "id": "faecedf0",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
    "source": [
     "Create a `.env` file in this folder using any editor."
    ]
@@ -160,7 +192,11 @@
   {
    "cell_type": "markdown",
    "id": "a880382a",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
    "source": [
     "Add secrets as key-value pairs in the `.env` file"
    ]
@@ -168,15 +204,25 @@
   {
    "cell_type": "markdown",
    "id": "04f00703",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
    "source": [
     "If you are using my OpenAI service use the following format:"
    ]
   },
   {
-   "cell_type": "markdown",
-   "id": "70e183f5",
-   "metadata": {},
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "33c7bf87",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
+   "outputs": [],
    "source": [
     "OPENAI_API_BASE=<my API base>\n",
     "OPENAI_API_KEY=<your API key to my service>"
@@ -185,7 +231,11 @@
   {
    "cell_type": "markdown",
    "id": "a952b103",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
    "source": [
     "If you are not using my OpenAI service then use the following format:"
    ]
@@ -194,7 +244,11 @@
    "cell_type": "code",
    "execution_count": null,
    "id": "7cf6ed7b",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
    "outputs": [],
    "source": [
     "OPENAI_API_KEY=<your OpenAI API key>"
@@ -203,7 +257,11 @@
   {
    "cell_type": "markdown",
    "id": "955963ed",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
    "source": [
     "Then, use the following code to load those secrets into this notebook:"
    ]
@@ -212,7 +270,11 @@
    "cell_type": "code",
    "execution_count": 47,
    "id": "fcadf45e",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
    "outputs": [
     {
      "data": {
@@ -234,7 +296,11 @@
   {
    "cell_type": "markdown",
    "id": "d3b6c394",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
    "source": [
     "#### Install Dependencies"
    ]
@@ -256,7 +322,11 @@
   {
    "cell_type": "markdown",
    "id": "f2ed966d",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
    "source": [
     "#### Let's make a function to wrap OpenAI functionality and write some basic tests"
    ]
@@ -264,7 +334,11 @@
   {
    "cell_type": "markdown",
    "id": "c1b09026",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
    "source": [
     "Start by simply seeing if we can make an API call"
    ]
@@ -273,7 +347,11 @@
    "cell_type": "code",
    "execution_count": 2,
    "id": "0abdd4e9",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
    "outputs": [
     {
      "name": "stdout",
@@ -300,7 +378,11 @@
   {
    "cell_type": "markdown",
    "id": "3f5d1530",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
    "source": [
     "Great! Now let's wrap that in a function"
    ]
@@ -340,7 +422,11 @@
   {
    "cell_type": "markdown",
    "id": "f4506fe8",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
    "source": [
     "Let's add some tests!\n",
     "\n",
@@ -381,7 +467,11 @@
   {
    "cell_type": "markdown",
    "id": "ed166c5b",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
    "source": [
     "But what if we do what to test the output of the LLM?\n",
     "\n",
@@ -402,7 +492,11 @@
    "cell_type": "code",
    "execution_count": 31,
    "id": "9c03b774",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
    "outputs": [],
    "source": [
     "import openai\n",
@@ -421,11 +515,29 @@
     "assert isinstance(get_ai_reply(\"hello\", model=\"gpt-3.5-turbo\"), str)"
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "15e09750",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
+   "outputs": [],
+   "source": [
+    "If we run this enough times we should see that the output for the bottom run is more inconsistent."
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 33,
    "id": "f5ed24da",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
    "outputs": [
     {
      "name": "stdout",
@@ -444,7 +556,11 @@
   {
    "cell_type": "markdown",
    "id": "ee320648",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
    "source": [
     "Ok great! Now, an LLM is no good to us if we can't _steer_ it.\n",
     "\n",
@@ -455,7 +571,11 @@
    "cell_type": "code",
    "execution_count": 37,
    "id": "295839a5",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
    "outputs": [],
    "source": [
     "import openai\n",
@@ -493,7 +613,11 @@
   {
    "cell_type": "markdown",
    "id": "fc9763d6",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
    "source": [
     "Let's see if we can get the LLM to follow instructions by adding instructions to the prompt.\n",
     "\n",
@@ -504,7 +628,11 @@
    "cell_type": "code",
    "execution_count": 36,
    "id": "c5e2a8b3",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
    "outputs": [
     {
      "name": "stdout",
@@ -521,32 +649,32 @@
   {
    "cell_type": "markdown",
    "id": "e59d3275",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
    "source": [
     "While the output is more or less controlled, the LLM responds with 'world.' or 'world'. While the word 'world' being in the string is pretty consistent, the punctuation is not.\n",
     "\n",
-    "We could do additional tweaking to the prompt to prevent that if we wanted.\n",
-    "\n",
-    "For now let's assume this is the best we can do.\n",
-    "\n",
-    "How do we write tests against this?\n",
-    "\n",
-    "Well, what do have high confidence won't change in the LLM output?\n",
+    "How do we write tests against this or have confidence with non-determinism?\n",
     "\n",
     "What is a test that we could write that:\n",
     "* would pass if the LLM outputs in a manner that is consistent with our expectations (and consistent with its own output)?\n",
     "* _we want to be true_ about our LLM system, and if it does not then we would want to know immediately and adjust our system?\n",
     "* if the prompt does not change, that our expectation holds true?\n",
-    "* someone changes the prompt in a way that would break the rest of the system, that we would want to prevent that from being merged without fixing the downstream effects?\n",
-    "\n",
-    "That might be a bunch of ways of saying the same thing but I hope you get the point."
+    "* someone changes the prompt in a way that would break the rest of the system, that we would want to prevent that from being merged without fixing the downstream effects?"
    ]
   },
   {
    "cell_type": "code",
    "execution_count": 38,
    "id": "d2b2bc15",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
    "outputs": [],
    "source": [
     "# non-deterministic tests\n",
@@ -559,7 +687,11 @@
   {
    "cell_type": "markdown",
    "id": "5e17eefe",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
    "source": [
     "Alright that worked!\n",
     "\n",
@@ -570,7 +702,11 @@
    "cell_type": "code",
    "execution_count": 39,
    "id": "4fd88c05",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
    "outputs": [],
    "source": [
     "import openai\n",
@@ -615,7 +751,11 @@
   {
    "cell_type": "markdown",
    "id": "e0a00cda",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
    "source": [
     "Now let's check that it works"
    ]
@@ -624,7 +764,11 @@
    "cell_type": "code",
    "execution_count": 42,
    "id": "977f99bd",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
    "outputs": [
     {
      "name": "stdout",
@@ -647,7 +791,11 @@
   {
    "cell_type": "markdown",
    "id": "2ad45888",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
    "source": [
     "Great! Now let's turn that into a test!"
    ]
@@ -656,7 +804,11 @@
    "cell_type": "code",
    "execution_count": 43,
    "id": "83aa7546",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
    "outputs": [],
    "source": [
     "system_message=\"The user will tell you their name. When asked, repeat their name back to them.\"\n",
@@ -672,7 +824,11 @@
   {
    "cell_type": "markdown",
    "id": "25e498c8",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
    "source": [
     "Alright here is our final function for integrating with OpenAI!"
    ]
@@ -681,7 +837,11 @@
    "cell_type": "code",
    "execution_count": null,
    "id": "bfc5cd86",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
    "outputs": [],
    "source": [
     "import openai\n",
@@ -736,7 +896,11 @@
    "cell_type": "code",
    "execution_count": 44,
    "id": "159eea8a",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
    "outputs": [
     {
      "name": "stdout",
@@ -753,14 +917,18 @@
   {
    "cell_type": "markdown",
    "id": "9de8f5da",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
    "source": [
     "In the next few lessons, we will be building a graphical user interface around this functionality so we can have a real conversational experience."
    ]
   }
  ],
  "metadata": {
-  "celltoolbar": "Raw Cell Format",
+  "celltoolbar": "Slideshow",
   "kernelspec": {
    "display_name": "Python 3 (ipykernel)",
    "language": "python",

04_creating_a_functional_conversational_chatbot.ipynb

@@ -345,18 +345,10 @@
     "# Set the share parameter to False, meaning the interface will not be publicly accessible\n",
     "get_chatbot_app().launch()"
    ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "ae24e591",
-   "metadata": {},
-   "outputs": [],
-   "source": []
   }
  ],
  "metadata": {
-  "celltoolbar": "Raw Cell Format",
+  "celltoolbar": "Slideshow",
   "kernelspec": {
    "display_name": "Python 3 (ipykernel)",
    "language": "python",

05_deploying_a_chatbot_to_the_web.ipynb

@@ -43,24 +43,36 @@
   },
   {
    "cell_type": "markdown",
-   "id": "7804a8ce",
-   "metadata": {},
+   "id": "60c8e7f6",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "-"
+    }
+   },
    "source": [
-    "#### Configuring the files required"
+    "Let's face it! Once we start building cool stuff we are going to want to show it off. It can take us < 10 minutes to deploy our chatbots and LLM applications when using Gradio!"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "60c8e7f6",
-   "metadata": {},
+   "id": "7804a8ce",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
    "source": [
-    "Let's face it! Once we start building cool stuff we are going to want to show it off. It can take us < 10 minutes to deploy our chatbots and LLM applications when using Gradio!"
+    "#### Configuring the files required"
    ]
   },
   {
    "cell_type": "markdown",
    "id": "54de0ddc",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "-"
+    }
+   },
    "source": [
     "Add a username and password for your app to your `.env` file. This will ensure that unauthorized users are not able to access LLM features. Use the following format:"
    ]
@@ -69,7 +81,11 @@
    "cell_type": "code",
    "execution_count": null,
    "id": "95dec7cf",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
    "outputs": [],
    "source": [
     "APP_USERNAME=<whatever username you want>\n",
@@ -79,24 +95,48 @@
   {
    "cell_type": "markdown",
    "id": "5072dc21",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
    "source": [
-    "Let's start by taking all of our necessary chatbot code into one file which we will name `app.py`. Run the following cell to automatically write it!"
+    "Let's start by taking all of our necessary chatbot code into one file which we will name `app.py`. Which I'll show you how to do in a second."
    ]
   },
   {
    "cell_type": "markdown",
    "id": "aacdcaaf",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
    "source": [
     "Take note that this code has been altered a little bit from the last chatbot example in order to add authentication."
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "78815586",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
+   "source": [
+    "In a Jupyter notebook, `%%writefile` is a magic command that tells the notebook to write the contents of the cell to a file. When you run the following cell, it will create (or overwrite) the specified file `app.py` in the current working directory and save the cell's content to that file. It won't actually run the code, just save it. This is useful when you want to save your code directly from a Jupyter notebook cell to a separate Python script file."
+   ]
+  },
   {
    "cell_type": "code",
-   "execution_count": 12,
+   "execution_count": 14,
    "id": "710b66f7",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
    "outputs": [
     {
      "name": "stdout",
@@ -242,7 +282,11 @@
   {
    "cell_type": "markdown",
    "id": "6d75af66",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
    "source": [
     "We will also need a `requirements.txt` file to store the list of the packages that HuggingFace needs to install to run our chatbot."
    ]
@@ -251,7 +295,11 @@
    "cell_type": "code",
    "execution_count": 13,
    "id": "14d0e434",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
    "outputs": [
     {
      "name": "stdout",
@@ -271,7 +319,11 @@
   {
    "cell_type": "markdown",
    "id": "4debec45",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
    "source": [
     "Now let's go ahead and commit our changes"
    ]
@@ -280,7 +332,11 @@
    "cell_type": "code",
    "execution_count": 9,
    "id": "14d42a96",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
    "outputs": [],
    "source": [
     "!git add app.py"
@@ -290,7 +346,11 @@
    "cell_type": "code",
    "execution_count": 8,
    "id": "d7c5b127",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
    "outputs": [],
    "source": [
     "!git add requirements.txt"
@@ -300,7 +360,11 @@
    "cell_type": "code",
    "execution_count": 11,
    "id": "18960d9f",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
    "outputs": [
     {
      "name": "stdout",
@@ -320,7 +384,11 @@
   {
    "cell_type": "markdown",
    "id": "09221ee0",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
    "source": [
     "#### Using HuggingFace Spaces"
    ]
@@ -328,7 +396,11 @@
   {
    "cell_type": "markdown",
    "id": "db789c94",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "-"
+    }
+   },
    "source": [
     "As mentioned before, HuggingFace is a free-to-use platform for hosting AI demos and apps. We will need to make a HuggingFace _Space_ for our chatbot."
    ]
@@ -336,31 +408,235 @@
   {
    "cell_type": "markdown",
    "id": "d9eedd10",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
+   "source": [
+    "First sign up for a free HuggingFace account [here](https://huggingface.co/join). "
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "00f537d9",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "-"
+    }
+   },
    "source": [
-    "First sign up for a free HuggingFace account [here](https://huggingface.co/join). After you sign up, create a new Space by clicking \"New Space\" on the navigation menu (press on your profile image)."
+    "<img src=\"images/signup.png\">"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "f5fc871f",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
+   "source": [
+    "After you sign up, create a new Space by clicking \"New Space\" on the navigation menu (press on your profile image)."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "f84e8451",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "-"
+    }
+   },
+   "source": [
+    "<img src=\"images/dropdown.png\">"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "85684a7b",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
+   "source": [
+    "Create a new space with the following settings"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "e6130b5d",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "-"
+    }
+   },
+   "source": [
+    "<img src=\"images/space-settings.png\">"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "897729d3",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
+   "source": [
+    "Take note of your HuggingFace Space URL"
    ]
   },
   {
    "cell_type": "markdown",
    "id": "3e042d24",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
    "source": [
     "#### Generate a HuggingFace Access Token"
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "67273b1a",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
+   "outputs": [],
+   "source": [
+    "In order to push our code to HuggingFace, we will need to authenticate with HuggingFace via an access token on Git.\n",
+    "\n",
+    "HuggingFace uses git to push a copy of your code to their servers for deployment."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "909216d4",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
+   "source": [
+    "Visit your Profile Settings and click 'Access Tokens'"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "fc1c1bec",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "-"
+    }
+   },
+   "source": [
+    "<img src=\"images/profile-settings.png\">"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "4af1eb8e",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
+   "source": [
+    "Create a new access token for your app with 'write' permissions."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "e4958b80",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "-"
+    }
+   },
+   "source": [
+    "<img src=\"images/access-tokens.png\">"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "230d1a32",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
+   "source": [
+    "<img src=\"images/new-access-token.png\">"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "f050684c",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
+   "source": [
+    "Copy the token to your clipboard!"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "aa776b02",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "-"
+    }
+   },
+   "source": [
+    "<img src=\"images/copy-token.png\">"
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "a7d0781d",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
    "source": [
     "#### Login to HuggingFace Hub"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "74518b91",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "-"
+    }
+   },
+   "source": [
+    "If you have ever used Heroku, HuggingFace is similar. The way you deploy is by pushing them a copy of your git repository.\n",
+    "\n",
+    "In order for us to do this, our local git will need to be able to authenticate with HuggingFace.\n",
+    "\n",
+    "Fortunately, theres an easy way we can authenticate and we can do it from inside this notebook."
+   ]
+  },
   {
    "cell_type": "markdown",
    "id": "eba83252",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
    "source": [
     "Install `huggingface_hub`"
    ]
@@ -369,7 +645,11 @@
    "cell_type": "code",
    "execution_count": 40,
    "id": "266bf481",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "-"
+    }
+   },
    "outputs": [],
    "source": [
     "!pip -q install --upgrade huggingface_hub"
@@ -378,7 +658,11 @@
   {
    "cell_type": "markdown",
    "id": "d50cd84b",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
    "source": [
     "Login to HuggingFace"
    ]
@@ -387,7 +671,11 @@
    "cell_type": "code",
    "execution_count": 6,
    "id": "53fd5037",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "-"
+    }
+   },
    "outputs": [
     {
      "name": "stdout",
@@ -408,7 +696,11 @@
   {
    "cell_type": "markdown",
    "id": "90f9bd4d",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
    "source": [
     "#### Now lets setup git and HuggingFace Spaces to work together and deploy"
    ]
@@ -416,7 +708,11 @@
   {
    "cell_type": "markdown",
    "id": "66468481",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
    "source": [
     "<span style=\"color:red\">REPLACE MY URL WITH YOURS</span>"
    ]
@@ -425,7 +721,11 @@
    "cell_type": "code",
    "execution_count": null,
    "id": "827a201d",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
    "outputs": [],
    "source": [
     "!git remote add huggingface https://huggingface.co/spaces/ericmichael/gradio-chatbot-demo"
@@ -434,7 +734,11 @@
   {
    "cell_type": "markdown",
    "id": "f8b3bb3d",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
    "source": [
     "Then force push to sync everything for the first time."
    ]
@@ -443,7 +747,11 @@
    "cell_type": "code",
    "execution_count": 12,
    "id": "86c9ee4e",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
    "outputs": [
     {
      "name": "stdout",
@@ -461,39 +769,158 @@
   },
   {
    "cell_type": "markdown",
-   "id": "3a353ebf",
-   "metadata": {},
+   "id": "48510bba",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
    "source": [
-    "That's it! 🎉 Check your HuggingFace Space URL to access your chatbot!"
+    "#### Now lets setup our secrets on our HuggingFace Space"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "4ab2d59b",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
+   "source": [
+    "We're so close, HuggingFace now has our app but it doesn't have any of our secrets! This means users can't authenticate with our HuggingFace app and our app can't authenticate with OpenAI!\n",
+    "\n",
+    "We will need to set our secrets manually on HuggingFace, which is a common practice so they don't end up accidentally leaked in version control."
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "5a54132b",
+   "id": "fec397c8",
    "metadata": {
     "slideshow": {
      "slide_type": "slide"
     }
    },
    "source": [
-    "## Deploying using FastAPI"
+    "Go to your HuggingFace Space and click 'Settings'"
    ]
   },
   {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "181dd4ad",
+   "cell_type": "markdown",
+   "id": "a7016df7",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "-"
+    }
+   },
+   "source": [
+    "<img src=\"images/repo.png\">"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "1e10cf24",
    "metadata": {
     "slideshow": {
      "slide_type": "slide"
     }
    },
-   "outputs": [],
-   "source": []
+   "source": [
+    "Scroll to 'Repository Secrets', yours will be empty for now but you will need to fill it in."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "d70ec1aa",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "-"
+    }
+   },
+   "source": [
+    "<img src=\"images/secrets.png\">"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "8c931425",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
+   "source": [
+    "We will need to provide:\n",
+    "\n",
+    "* `OPENAI_API_BASE` - if you are using my OpenAI service, don't add this one if you are paying for OpenAI yourself\n",
+    "* `OPENAI_API_KEY` - either the key from my service or from OpenAI directly\n",
+    "* `APP_USERNAME` - the username you want people to sign in with\n",
+    "* `APP_PASSWORD` - the username you want people to sign in with"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "17a2e7c5",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
+   "source": [
+    "Example"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "7132a08f",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "-"
+    }
+   },
+   "source": [
+    "<img src=\"images/add-secret.png\">"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "4405e200",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
+   "source": [
+    "Restart your HuggingFace Space"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "0718c510",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "-"
+    }
+   },
+   "source": [
+    "<img src=\"images/restart.png\">"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "3a353ebf",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "-"
+    }
+   },
+   "source": [
+    "That's it! 🎉 Check your HuggingFace Space URL to access your chatbot!"
+   ]
   }
  ],
  "metadata": {
-  "celltoolbar": "Raw Cell Format",
+  "celltoolbar": "Slideshow",
   "kernelspec": {
    "display_name": "Python 3 (ipykernel)",
    "language": "python",

06_designing_effective_prompts.ipynb

@@ -41,7 +41,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 140,
+   "execution_count": 2,
    "id": "895d5bf1",
    "metadata": {
     "hideCode": false,
@@ -1484,90 +1484,377 @@
     }
    },
    "source": [
-    "## Designing Effective Prompts: Text Generation, Summarization, Translation"
+    "## Additional Examples"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 18,
-   "id": "73063bc4",
-   "metadata": {},
-   "outputs": [],
+   "execution_count": 13,
+   "id": "8e09db5c",
+   "metadata": {
+    "hideCode": true,
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/html": [
+       "\n",
+       "    <table style=\"font-size: 100%;\">\n",
+       "        <tr style=\"background: none;\">\n",
+       "            <td style=\"vertical-align: top;text-align:left;\" width=\"45%\">\n",
+       "                <h4>Prompt</h4><div style='color:red;padding-top:10px;font-weight:300;font-style: italic;'>Bad, ambigous without context</div>\n",
+       "                <div style=\"line-height:1.5;border:1px solid #e9ecef;margin-top:9px;color:#495057;background-color:#ffe69c;padding: 12px;border-radius: 8px;\">\n",
+       "                    You are an AI assistant that can provide translations for words and sentences.<br/><br/>Translate that word or sentence to Spanish, do not provide unnecessary commentary or punctuation.\n",
+       "                </div>\n",
+       "            </td>\n",
+       "            <td width=\"10%\"></td>\n",
+       "            <td style=\"vertical-align: top;text-align:right;\" width=\"45%\">\n",
+       "                <h4>User Input</h4>\n",
+       "                <div style=\"line-height:1.5;margin-top:9px;color:white;background-color:#0b93f6;padding: 12px;border-radius: 8px;\">\n",
+       "                    I saw the bat.\n",
+       "                </div>\n",
+       "                <h4>LLM Output</h4>\n",
+       "                <div style=\"line-height:1.5;margin-top:9px;color:black;background-color:#e5e5ea;padding: 12px;border-radius: 8px;\">\n",
+       "                    Vi el murciélago.\n",
+       "                </div>\n",
+       "            </td>\n",
+       "        </tr>\n",
+       "    </table>\n",
+       "    "
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
    "source": [
-    "import openai\n",
+    "prompt = \"\"\"\n",
+    "You are an AI assistant that can provide translations for words and sentences.\n",
     "\n",
-    "# Define a function to get the AI's reply using the OpenAI API\n",
-    "def simple_reply(prompt, message):\n",
-    "    # Initialize the messages list with the system message\n",
-    "    messages = [{\"role\": \"system\", \"content\": prompt}]\n",
-    "    \n",
-    "    # Add the user's message to the messages list\n",
-    "    messages += [{\"role\": \"user\", \"content\": message}]\n",
-    "    \n",
-    "    # Make an API call to the OpenAI ChatCompletion endpoint with the model and messages\n",
-    "    completion = openai.ChatCompletion.create(\n",
-    "        model=\"gpt-3.5-turbo\",\n",
-    "        messages=messages\n",
-    "    )\n",
-    "    \n",
-    "    # Extract and return the AI's response from the API response\n",
-    "    return completion.choices[0].message.content"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "1a2e4dbb",
-   "metadata": {},
-   "source": [
-    "#### Idea: Entrepreneur AI Prompt"
+    "Translate that word or sentence to Spanish, do not provide unnecessary commentary or punctuation.\n",
+    "\"\"\"\n",
+    "\n",
+    "user_message = \"I saw the bat.\"\n",
+    "\n",
+    "display_chat(prompt, user_message, label=\"Bad, ambigous without context\")"
    ]
   },
   {
-   "cell_type": "markdown",
-   "id": "afb1f550",
-   "metadata": {},
+   "cell_type": "code",
+   "execution_count": 15,
+   "id": "cee637d0",
+   "metadata": {
+    "hideCode": true,
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
+   "outputs": [
+    {
+     "data": {
+      "text/html": [
+       "\n",
+       "    <table style=\"font-size: 100%;\">\n",
+       "        <tr style=\"background: none;\">\n",
+       "            <td style=\"vertical-align: top;text-align:left;\" width=\"45%\">\n",
+       "                <h4>Prompt</h4><div style='color:green;padding-top:10px;font-weight:300;font-style: italic;'>Good, provides the context needed for translation</div>\n",
+       "                <div style=\"line-height:1.5;border:1px solid #e9ecef;margin-top:9px;color:#495057;background-color:#ffe69c;padding: 12px;border-radius: 8px;\">\n",
+       "                    You are an AI assistant that can provide translations for words and sentences.<br/><br/>Context: A detective is looking for a murder weapon and is speaking to a witness.<br/><br/>Translate that word or sentence to Spanish, do not provide unnecessary commentary or punctuation.\n",
+       "                </div>\n",
+       "            </td>\n",
+       "            <td width=\"10%\"></td>\n",
+       "            <td style=\"vertical-align: top;text-align:right;\" width=\"45%\">\n",
+       "                <h4>User Input</h4>\n",
+       "                <div style=\"line-height:1.5;margin-top:9px;color:white;background-color:#0b93f6;padding: 12px;border-radius: 8px;\">\n",
+       "                    I saw the bat.\n",
+       "                </div>\n",
+       "                <h4>LLM Output</h4>\n",
+       "                <div style=\"line-height:1.5;margin-top:9px;color:black;background-color:#e5e5ea;padding: 12px;border-radius: 8px;\">\n",
+       "                    Vi el bate.\n",
+       "                </div>\n",
+       "            </td>\n",
+       "        </tr>\n",
+       "    </table>\n",
+       "    "
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
    "source": [
-    "Problem: Coming up with _good_ app ideas that are simple enough for a solo developer to pursue.\n",
+    "prompt = \"\"\"\n",
+    "You are an AI assistant that can provide translations for words and sentences.\n",
+    "\n",
+    "Context: A detective is looking for a murder weapon and is speaking to a witness.\n",
     "\n",
-    "Challenges:\n",
-    "* Many college students \n"
+    "Translate that word or sentence to Spanish, do not provide unnecessary commentary or punctuation.\n",
+    "\"\"\"\n",
+    "\n",
+    "user_message = \"I saw the bat.\"\n",
+    "\n",
+    "display_chat(prompt, user_message, label=\"Good, provides the context needed for translation\")"
    ]
   },
   {
-   "cell_type": "markdown",
-   "id": "e9a7bff9",
+   "cell_type": "code",
+   "execution_count": 17,
+   "id": "5b59cc98",
    "metadata": {
+    "hideCode": true,
     "slideshow": {
      "slide_type": "slide"
     }
    },
+   "outputs": [
+    {
+     "data": {
+      "text/html": [
+       "\n",
+       "    <table style=\"font-size: 100%;\">\n",
+       "        <tr style=\"background: none;\">\n",
+       "            <td style=\"vertical-align: top;text-align:left;\" width=\"45%\">\n",
+       "                <h4>Prompt</h4><div style='color:red;padding-top:10px;font-weight:300;font-style: italic;'>Bad</div>\n",
+       "                <div style=\"line-height:1.5;border:1px solid #e9ecef;margin-top:9px;color:#495057;background-color:#ffe69c;padding: 12px;border-radius: 8px;\">\n",
+       "                    You are an AI assistant that can clean up auto-generated text transcriptions from YouTube videos.<br/><br/>When the user submits text.<br/><br/>Reply only with the cleaned up version of the text.\n",
+       "                </div>\n",
+       "            </td>\n",
+       "            <td width=\"10%\"></td>\n",
+       "            <td style=\"vertical-align: top;text-align:right;\" width=\"45%\">\n",
+       "                <h4>User Input</h4>\n",
+       "                <div style=\"line-height:1.5;margin-top:9px;color:white;background-color:#0b93f6;padding: 12px;border-radius: 8px;\">\n",
+       "                    so i have a new series coming out<br/>on kgard which is my favorite way to<br/>approach<br/>guard against an opponent who is<br/>kneeling for open guard okay this is<br/>going to be part of<br/>three releases i have coming out on open<br/>guard there&#x27;s keyguard<br/>there&#x27;s de la hever and wader and then<br/>there&#x27;s leg entanglements and reverse<br/>telehebera<br/>okay so for keigar if i&#x27;ve got jorge<br/>here<br/>usually usually against someone who&#x27;s on<br/>their knees the first entry point<br/>for me is to look for keigar\n",
+       "                </div>\n",
+       "                <h4>LLM Output</h4>\n",
+       "                <div style=\"line-height:1.5;margin-top:9px;color:black;background-color:#e5e5ea;padding: 12px;border-radius: 8px;\">\n",
+       "                    &quot;I have a new series coming out on Keigar, my favorite way to guard against an opponent who is kneeling for open guard. This is part of three releases I have coming out on open guard, which include De La Hever and Wader, and leg entanglements and reverse telehebera. When using Keigar, my first entry point against someone on their knees is to look for Keigar.&quot;\n",
+       "                </div>\n",
+       "            </td>\n",
+       "        </tr>\n",
+       "    </table>\n",
+       "    "
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
    "source": [
-    "## Designing Effective Prompts: Code Generation, Completion, and Bug Detection"
+    "prompt = \"\"\"\n",
+    "You are an AI assistant that can clean up auto-generated text transcriptions from YouTube videos.\n",
+    "\n",
+    "When the user submits text.\n",
+    "\n",
+    "Reply only with the cleaned up version of the text.\n",
+    "\"\"\"\n",
+    "\n",
+    "user_message = \"\"\"\n",
+    "so i have a new series coming out\n",
+    "on kgard which is my favorite way to\n",
+    "approach\n",
+    "guard against an opponent who is\n",
+    "kneeling for open guard okay this is\n",
+    "going to be part of\n",
+    "three releases i have coming out on open\n",
+    "guard there's keyguard\n",
+    "there's de la hever and wader and then\n",
+    "there's leg entanglements and reverse\n",
+    "telehebera\n",
+    "okay so for keigar if i've got jorge\n",
+    "here\n",
+    "usually usually against someone who's on\n",
+    "their knees the first entry point\n",
+    "for me is to look for keigar\n",
+    "\"\"\"\n",
+    "\n",
+    "display_chat(prompt, user_message, label=\"Bad\")"
    ]
   },
   {
-   "cell_type": "markdown",
-   "id": "29b87563",
+   "cell_type": "code",
+   "execution_count": 30,
+   "id": "dc452072",
    "metadata": {
+    "hideCode": true,
     "slideshow": {
      "slide_type": "slide"
     }
    },
+   "outputs": [
+    {
+     "data": {
+      "text/html": [
+       "\n",
+       "    <table style=\"font-size: 100%;\">\n",
+       "        <tr style=\"background: none;\">\n",
+       "            <td style=\"vertical-align: top;text-align:left;\" width=\"45%\">\n",
+       "                <h4>Prompt</h4><div style='color:green;padding-top:10px;font-weight:300;font-style: italic;'>Good, this can be used as a subroutine for LLM chains.</div>\n",
+       "                <div style=\"line-height:1.5;border:1px solid #e9ecef;margin-top:9px;color:#495057;background-color:#ffe69c;padding: 12px;border-radius: 8px;\">\n",
+       "                    You are an AI assistant that can aid humans in assist in cleaning auto-generated text transcriptions from YouTube videos.<br/><br/>When the user submits the title of the YouTube video.<br/><br/>Reply with unique vocabulary words or short phrases that relate to that topic that an auto-generated transcription is likely to misspell due to the words or phrases being unique nomenclature to that topic.<br/><br/>Reply only with the vocabulary list.<br/><br/>You must provide 50 words or phrases.<br/><br/>Use the following format:<br/>- &lt;word or phrase&gt;<br/>- &lt;word or phrase&gt;<br/>- &lt;word or phrase&gt;<br/>- &lt;word or phrase&gt;<br/>- &lt;word or phrase&gt;<br/>- &lt;word or phrase&gt;<br/>- &lt;word or phrase&gt;<br/>...<br/>- &lt;word or phrase&gt;\n",
+       "                </div>\n",
+       "            </td>\n",
+       "            <td width=\"10%\"></td>\n",
+       "            <td style=\"vertical-align: top;text-align:right;\" width=\"45%\">\n",
+       "                <h4>User Input</h4>\n",
+       "                <div style=\"line-height:1.5;margin-top:9px;color:white;background-color:#0b93f6;padding: 12px;border-radius: 8px;\">\n",
+       "                    How To Do The Perfect BJJ K Guard by Lachlan Giles\n",
+       "                </div>\n",
+       "                <h4>LLM Output</h4>\n",
+       "                <div style=\"line-height:1.5;margin-top:9px;color:black;background-color:#e5e5ea;padding: 12px;border-radius: 8px;\">\n",
+       "                    - BJJ <br/>- K Guard<br/>- Lachlan Giles<br/>- Brazilian Jiu-Jitsu<br/>- Grappling<br/>- Submission<br/>- Martial arts<br/>- Gi <br/>- No Gi<br/>- Ground Game<br/>- Sweep<br/>- Choke<br/>- Armbar<br/>- Triangle<br/>- Kimura<br/>- Omoplata<br/>- Guard Pass<br/>- Open Guard<br/>- Closed Guard<br/>- Half Guard<br/>- Full Guard<br/>- Mount<br/>- Back Control<br/>- Side Control<br/>- Knee-on-Belly <br/>- Spider Guard<br/>- De La Riva Guard<br/>- X Guard<br/>- Butterfly Guard<br/>- Sprawl<br/>- Hip Escape<br/>- Scissor Sweep<br/>- Butterfly Sweep<br/>- Single Leg Takedown<br/>- Double Leg Takedown<br/>- Shoulder Pressure<br/>- Guard Retention<br/>- Guard Recovery<br/>- Submission Defense<br/>- Positional Control<br/>- Joint Lock<br/>- Wrist Lock<br/>- Rear Naked Choke<br/>- Guillotine Choke<br/>- Arm Triangle Choke<br/>- Baseball Bat Choke<br/>- Bow and Arrow Choke<br/>- Triangle Choke Defense<br/>- Mount Escape<br/>- Collar Drag<br/>- Foot Sweep<br/>- Balloon Sweep.\n",
+       "                </div>\n",
+       "            </td>\n",
+       "        </tr>\n",
+       "    </table>\n",
+       "    "
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
    "source": [
-    "## Designing Effective Prompts: Chatbots, Conversational AI, and Customer Support Automation"
+    "prompt = \"\"\"\n",
+    "You are an AI assistant that can aid humans in assist in cleaning auto-generated text transcriptions from YouTube videos.\n",
+    "\n",
+    "When the user submits the title of the YouTube video.\n",
+    "\n",
+    "Reply with unique vocabulary words or short phrases that relate to that topic that an auto-generated transcription is likely to misspell due to the words or phrases being unique nomenclature to that topic.\n",
+    "\n",
+    "Reply only with the vocabulary list.\n",
+    "\n",
+    "You must provide 50 words or phrases.\n",
+    "\n",
+    "Use the following format:\n",
+    "- <word or phrase>\n",
+    "- <word or phrase>\n",
+    "- <word or phrase>\n",
+    "- <word or phrase>\n",
+    "- <word or phrase>\n",
+    "- <word or phrase>\n",
+    "- <word or phrase>\n",
+    "...\n",
+    "- <word or phrase>\n",
+    "\"\"\"\n",
+    "\n",
+    "user_message = \"\"\"\n",
+    "How To Do The Perfect BJJ K Guard by Lachlan Giles\n",
+    "\"\"\"\n",
+    "\n",
+    "display_chat(prompt, user_message, label=\"Good, this can be used as a subroutine for LLM chains.\")"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": null,
-   "id": "181dd4ad",
+   "execution_count": 29,
+   "id": "90d9439a",
    "metadata": {
+    "hideCode": true,
     "slideshow": {
      "slide_type": "slide"
     }
    },
-   "outputs": [],
-   "source": []
+   "outputs": [
+    {
+     "data": {
+      "text/html": [
+       "\n",
+       "    <table style=\"font-size: 100%;\">\n",
+       "        <tr style=\"background: none;\">\n",
+       "            <td style=\"vertical-align: top;text-align:left;\" width=\"45%\">\n",
+       "                <h4>Prompt</h4><div style='color:green;padding-top:10px;font-weight:300;font-style: italic;'>Good, includes context about the topic and a known list of vocabulary words. Can the LLM come up with those words though?</div>\n",
+       "                <div style=\"line-height:1.5;border:1px solid #e9ecef;margin-top:9px;color:#495057;background-color:#ffe69c;padding: 12px;border-radius: 8px;\">\n",
+       "                    You are an AI assistant that can clean up auto-generated text transcriptions from YouTube videos.<br/><br/>When the user submits text.<br/><br/>Reply only with the cleaned up version of the text.<br/><br/>If you are about to output something that looks like it is misspelled check first against the list of unique vocabulary<br/>and also think about the context you know about the topic.<br/><br/>If you output a word or phrase it should be because you have context that associates it with the topic.<br/><br/>Topic: Brazilian Jiu-Jitsu<br/>Unique Vocabulary:<br/>- K-Guard<br/>- Half-Guard<br/>- Single Leg X<br/>- Single Leg<br/>- Z-Guard<br/>- De La Riva<br/>- Reverse De Riva<br/>- Mount<br/>- Back Control<br/>- Ezekiel<br/>- Kimura<br/>- Waiter\n",
+       "                </div>\n",
+       "            </td>\n",
+       "            <td width=\"10%\"></td>\n",
+       "            <td style=\"vertical-align: top;text-align:right;\" width=\"45%\">\n",
+       "                <h4>User Input</h4>\n",
+       "                <div style=\"line-height:1.5;margin-top:9px;color:white;background-color:#0b93f6;padding: 12px;border-radius: 8px;\">\n",
+       "                    so i have a new series coming out<br/>on kgard which is my favorite way to<br/>approach<br/>guard against an opponent who is<br/>kneeling for open guard okay this is<br/>going to be part of<br/>three releases i have coming out on open<br/>guard there&#x27;s keyguard<br/>there&#x27;s de la hever and wader and then<br/>there&#x27;s leg entanglements and reverse<br/>telehebera<br/>okay so for keigar if i&#x27;ve got jorge<br/>here<br/>usually usually against someone who&#x27;s on<br/>their knees the first entry point<br/>for me is to look for keigar\n",
+       "                </div>\n",
+       "                <h4>LLM Output</h4>\n",
+       "                <div style=\"line-height:1.5;margin-top:9px;color:black;background-color:#e5e5ea;padding: 12px;border-radius: 8px;\">\n",
+       "                    &quot;I have a new series coming out on K-Guard, which is my favorite way to approach open guard against an opponent who is kneeling. This is going to be part of three releases I have coming out on open guard. There are K-Guard, De La Riva, and Waiter, and then there&#x27;s leg entanglements and reverse De La Riva. For K-Guard, if I&#x27;ve got Jorge here, usually against someone who&#x27;s on their knees, the first entry point for me is to look for K-Guard.&quot;\n",
+       "                </div>\n",
+       "            </td>\n",
+       "        </tr>\n",
+       "    </table>\n",
+       "    "
+      ],
+      "text/plain": [
+       "<IPython.core.display.HTML object>"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "prompt = \"\"\"\n",
+    "You are an AI assistant that can clean up auto-generated text transcriptions from YouTube videos.\n",
+    "\n",
+    "When the user submits text.\n",
+    "\n",
+    "Reply only with the cleaned up version of the text.\n",
+    "\n",
+    "If you are about to output something that looks like it is misspelled check first against the list of unique vocabulary\n",
+    "and also think about the context you know about the topic.\n",
+    "\n",
+    "If you output a word or phrase it should be because you have context that associates it with the topic.\n",
+    "\n",
+    "Topic: Brazilian Jiu-Jitsu\n",
+    "Unique Vocabulary:\n",
+    "- K-Guard\n",
+    "- Half-Guard\n",
+    "- Single Leg X\n",
+    "- Single Leg\n",
+    "- Z-Guard\n",
+    "- De La Riva\n",
+    "- Reverse De Riva\n",
+    "- Mount\n",
+    "- Back Control\n",
+    "- Ezekiel\n",
+    "- Kimura\n",
+    "- Waiter\n",
+    "\"\"\"\n",
+    "\n",
+    "user_message = \"\"\"\n",
+    "so i have a new series coming out\n",
+    "on kgard which is my favorite way to\n",
+    "approach\n",
+    "guard against an opponent who is\n",
+    "kneeling for open guard okay this is\n",
+    "going to be part of\n",
+    "three releases i have coming out on open\n",
+    "guard there's keyguard\n",
+    "there's de la hever and wader and then\n",
+    "there's leg entanglements and reverse\n",
+    "telehebera\n",
+    "okay so for keigar if i've got jorge\n",
+    "here\n",
+    "usually usually against someone who's on\n",
+    "their knees the first entry point\n",
+    "for me is to look for keigar\n",
+    "\"\"\"\n",
+    "\n",
+    "display_chat(prompt, user_message, label=\"Good, includes context about the topic and a known list of vocabulary words. Can the LLM come up with those words though?\")"
+   ]
   }
  ],
  "metadata": {

07_software_engineering_applied_to_llms.ipynb

@@ -18,217 +18,526 @@
   {
    "cell_type": "markdown",
    "id": "02d71e5e",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
    "source": [
-    "## Concerns with Quality and Performance Issues of LLM Integrated Apps"
+    "#### 🔍 Concerns with Quality and Performance Issues of LLM Integrated Apps"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "60fef658",
+   "id": "449db7d4",
    "metadata": {
     "slideshow": {
-     "slide_type": "slide"
+     "slide_type": "fragment"
+    }
+   },
+   "source": [
+    "❓ **Accuracy** - How do we boost confidence, correctness, and quality in outputs?"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "d86252a4",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
+   "source": [
+    "❓ **Validation & Verification** - How do we know we built the right thing? How do we know we built the thing right?"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "681ffe8e",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
+   "source": [
+    "❓ **Bias and Ethics** - How do we minimize harmful output?"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "441d106d",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
     }
    },
    "source": [
-    "* Applications depend on external APIs which has issues with flakiness and pricing, how do we avoid hitting APIs in testing?\n",
-    "* Responses may not be correct or accurate, how do we increase confidence in result?\n",
-    "* Responses may be biased or unethical or unwanted output, how do we stop this type of output?\n",
-    "* User requests could be unethical or unwanted input, how do we filter this type of input?\n"
+    "❓ **How** does traditional software engineering practices relate to LLM prompts and systems?"
    ]
   },
   {
    "cell_type": "markdown",
    "id": "5ef5865b",
-   "metadata": {},
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
    "source": [
     "## Lessons from Software Engineering Applied to LLMS:"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "2fc1b19a",
+   "id": "651ed117",
    "metadata": {
     "slideshow": {
      "slide_type": "slide"
     }
    },
    "source": [
-    "#### Prototyping\n",
-    "* Develop prompt prototypes early when working with customers or stakeholders, it is fast and cheap to test that the idea will work.\n",
-    "* Test against realistic examples, early. Fail fast and iterate quickly.\n",
-    "* Make a plan for how you will source dynamic data. If there is no path, the project is dead in the water."
+    "### 🎯 Agile & Iterative Design:"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "d784a289",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
+   "source": [
+    "🔸 **Rapid prompt prototyping** - Engage customers/stakeholders early on validate you are building the right prompts"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "907c5e83",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
+   "source": [
+    "🔸 **Real-world examples** - Test and iterate quickly against examples that simulate real-world inputs from users. Consider edge-cases early"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "3392caa7",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
+   "source": [
+    "🔸 **Collaborate** - Discuss and plan ahead how you might get access to external data sources or integrate into other systems"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "2528a3c9",
+   "id": "4aa1c373",
    "metadata": {
     "slideshow": {
      "slide_type": "slide"
     }
    },
    "source": [
-    "#### Testing\n",
-    "* Unit test prompts using traditional methods to increase confidence.\n",
-    "* Unit test your prompts using LLMs to increase confidence.\n",
-    "* Write tests that handle API errors or bad output (malformed, incorrect, unethical).\n",
-    "* Use 'mocking' in integration tests to avoid unnecessary calls to APIs, flakiness, and unwanted charges."
+    "### 📏 Testing:"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "d9cdafd2",
+   "id": "d40f58cf",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
+   "source": [
+    "✅ Unit test your prompts with traditional test frameworks"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "1e0c1912",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
+   "source": [
+    "✅ Leverage LLMs for non-deterministic unit testing and generating example data"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "98b83f99",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
+   "source": [
+    "✅ Handle API errors, bad output, and harmful output as part of your testing suite, CI practices, and team workflow"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "be00ab2a",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
+   "source": [
+    "✅ Use mocking to prevent unwanted API calls in integration tests (and save money!)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "adb5e77a",
    "metadata": {
     "slideshow": {
      "slide_type": "slide"
     }
    },
    "source": [
-    "#### Handling Bad Output\n",
-    "* Develop 'retry' mechanisms when you get unwanted output.\n",
-    "* Develop specific prompts for different 'retry' conditions. Include the context, what went wrong, and what needs to be fixed.\n",
-    "* Consider adding logging to your app to keep track of how often your app gets bad output."
+    "### 🔄 Handling Bad Output:"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "aceb00b5",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
+   "source": [
+    "💡 **Error handing** - Recover from unwanted output or incorrect output with robust retry mechanisms"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "252c7d5b",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
+   "source": [
+    "💡 **Customized retry prompts** - Guide the LLM with custom prompts that include the desired goal, the output, and the error."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "374eb0be",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
+   "source": [
+    "💡 **Logging and Monitoring** - Track output quality, malicious input, and harmful output."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "95af337f",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "slide"
+    }
+   },
+   "source": [
+    "### 📚 Template Languages & Version Control:"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "2972ba3f",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
+   "source": [
+    "📝 **Dynamic template languages** - Use templating engines like ERB, Handlebars, etc for dynamically building prompts and leveraging existing testing tools"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "4f44c1c4",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
+   "source": [
+    "📝 **Version control** - Manage prompt templates in the app's repo"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "642bd8ae",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
+   "source": [
+    "📝 **Team Policy / Code Review** - Develop expectations and team practices around managing changes to prompts"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "8f7de0be",
+   "id": "e91111a4",
    "metadata": {
     "slideshow": {
      "slide_type": "slide"
     }
    },
    "source": [
-    "#### Template Languages and Version Control\n",
-    "* Consider writing your prompt templates in dynamic template languages like ERB, Handlebars, etc.\n",
-    "* Keep prompt templates and prompts in version control in your app's repo.\n",
-    "* Write tests for handling template engine errors."
+    "### 💉 Prompt Injection/Leakage:"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "db84eff5",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
+   "source": [
+    "🔐 Test for prompt injection attacks"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "99a6f711",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
+   "source": [
+    "🔐 Validate input at UI and LLM level"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "df148645",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
+   "source": [
+    "🔐 Use LLMs to analyze input/output similarity to known prompt injection attacks and/or the prompt itself"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "d6885dd4",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
+   "source": [
+    "🔐 Implement anomaly detection & incident response"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "3987a54c",
+   "id": "e6b3ef98",
    "metadata": {
     "slideshow": {
      "slide_type": "slide"
     }
    },
    "source": [
-    "#### Prompt Injection/Leakage\n",
-    "* User-facing prompts should be tested against prompt injection attacks\n",
-    "* Validate input at the UI and LLM level\n",
-    "* Consider using an LLM to check if an output is similar to the prompt\n",
-    "* Have mechanisms for anomaly detection and incident response"
+    "### 🔒 Security (Do Not):"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "2b79206c",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
+   "source": [
+    "🚫 Avoid storing API keys in app code, binaries, or metadata files"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "a0e0c388",
+   "id": "2b33263f",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
+   "source": [
+    "🚫 Give LLM access to tools with higher privileges than the users themselves!"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "1b4e2040",
    "metadata": {
     "slideshow": {
      "slide_type": "slide"
     }
    },
    "source": [
-    "#### Security\n",
-    "* **Do not:** store API keys in application code as strings, encrypted or not.\n",
-    "* **Do not:** store API keys in compiled binaries distributed to users.\n",
-    "* **Do not:** store API keys in metadeta files bundled with your application.\n",
-    "* **Do:** store API keys in environment variables or cloud secrets.\n",
-    "* **Do:** store API keys in a `.env` file that is blocked from version control. (Ideally these are encrypted with a secret that is not in version control, but that is beyond the scope of today's discussion.)\n",
-    "* **Do:** create an intermediate web app (or API) with authentication/authorization that delegates requests to LLMs at run-time for use in front-end applications.\n",
-    "* **Do:** if your front-end application does not have user accounts, consider implementing guest or anonymous accounts and expiring or rotating keys.\n",
-    "* **Do:** when allowing LLMs to use tools, consider designing systems to pass-through user ids to tools so that they tools operate at the same level of access as the end-user."
+    "### 🔒 Security (Do):"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "9d3ef132",
-   "metadata": {},
+   "id": "5cdf282f",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
    "source": [
-    "## Production Deployment Considerations"
+    "✅ Store API keys in environment variables or cloud secrets"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "a480904f",
-   "metadata": {},
+   "id": "8093990a",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
    "source": [
-    "* Wrap LLM features as web service or API. Don't give out your OpenAI keys directly in distributed software.\n",
-    "    - For example: Django, Flask, FastAPI, Express.js, Sinatra, Ruby on Rails"
+    "✅ Block files containing secrets from entering version control by adding them to `.gitignore`"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "e6274d71",
-   "metadata": {},
+   "id": "8b19f8e4",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
    "source": [
-    "* Consider whether there are any regulations that might impact how you handle data, such as GDPR and HIPAA.\n",
-    "    - Regulation may require specific data handling and storage practices.\n",
-    "    - Cloud providers may offer compliance certifications and assessment tools.\n",
-    "    - On-prem deployments can provide more control of data storage and processing, but might require more resources (hardware, people, software) for management and maintenance\n",
-    "    - Cloud providers like Azure have great tools like Azure Defender for Cloud and Microsoft Purview for managing compliance"
+    "✅ Use intermediate web apps/APIs with authentication/authorization for accessing LLM features"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "2f9b5cf9",
-   "metadata": {},
+   "id": "5ce316c7",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "fragment"
+    }
+   },
    "source": [
-    "- Using Cloud Services vs On-Prem\n",
-    "    - Cloud services offer many advantages such as scalability, flexibilitiy, cost-effectiveness, and ease of management.\n",
-    "    - Easy to spin up resources and scale based on demand, without worrying about infrastructure or maintenance.\n",
-    "    - Wide range of tools: performance optimization, monitoring, security, reliability."
+    "✅ Implement transparent guest/anonymous accounts & key rotation when apps don't require authentication to use LLM features"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "17111661",
-   "metadata": {},
+   "id": "9d3ef132",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "skip"
+    }
+   },
    "source": [
-    "- Container-based Architecture\n",
-    "    - Containerization is a lightweight virtualization method that packages an application and its dependencies into a single, portable unit called a container.\n",
-    "    - Containers can run consistently across different environments, making it easier to develop, test, and deploy applications. \n",
-    "    - Containerization is useful when you need to ensure consistent behavior across various platforms, simplify deployment and scaling, and improve resource utilization.\n",
-    "    - Common tools for deploying container-based architecture are Docker and Kubernetes."
+    "## Production Deployment Considerations"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "56890eec",
-   "metadata": {},
+   "id": "a480904f",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "skip"
+    }
+   },
+   "source": [
+    "#### Wrap LLM features as web service or API. Don't give out your OpenAI keys directly in distributed software.\n",
+    "* For example: Django, Flask, FastAPI, Express.js, Sinatra, Ruby on Rails"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "e6274d71",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "skip"
+    }
+   },
    "source": [
-    "- Serverless Architectures\n",
-    "    - Serverless architectures are a cloud computing model where the cloud provider manages the infrastructure and automatically allocates resources based on the application's needs.\n",
-    "    - Developers only need to focus on writing code, and the provider takes care of scaling, patching, and maintaining the underlying infrastructure. \n",
-    "    - Serverless architectures can be useful when you want to reduce operational overhead, build event-driven applications, and optimize costs by paying only for the resources you actually use.\n",
-    "    - Common tools to build serverless applications and APIs include Azure Functions, AWS Lambda, and Google Cloud Functions."
+    "#### Consider whether there are any regulations that might impact how you handle data, such as GDPR and HIPAA.\n",
+    "- Regulation may require specific data handling and storage practices.\n",
+    "- Cloud providers may offer compliance certifications and assessment tools.\n",
+    "- On-prem deployments can provide more control of data storage and processing, but might require more resources (hardware, people, software) for management and maintenance\n",
+    "- Cloud providers like Azure have great tools like Azure Defender for Cloud and Microsoft Purview for managing compliance"
    ]
   },
   {
    "cell_type": "markdown",
-   "id": "b390fa49",
-   "metadata": {},
+   "id": "2f9b5cf9",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "skip"
+    }
+   },
    "source": [
-    "- HuggingFace\n",
-    "    - Platforms like HuggingFace provide an ecosystem for sharing, collaborating, and deploying AI models, including LLMs. \n",
-    "    - They offer pre-trained models, tools, and APIs that simplify the development and integration of AI-powered applications. \n",
-    "    - These platforms can be useful when you want to leverage existing models, collaborate with the AI community, and streamline the deployment process for your LLM-based applications."
+    "#### Using Cloud Services vs On-Prem\n",
+    "- Cloud services offer many advantages such as scalability, flexibilitiy, cost-effectiveness, and ease of management.\n",
+    "- Easy to spin up resources and scale based on demand, without worrying about infrastructure or maintenance.\n",
+    "- Wide range of tools: performance optimization, monitoring, security, reliability."
    ]
   },
   {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "72115a67",
-   "metadata": {},
-   "outputs": [],
-   "source": []
+   "cell_type": "markdown",
+   "id": "17111661",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "skip"
+    }
+   },
+   "source": [
+    "#### Container-based Architecture\n",
+    "- Containerization is a lightweight virtualization method that packages an application and its dependencies into a single, portable unit called a container.\n",
+    "- Containers can run consistently across different environments, making it easier to develop, test, and deploy applications. \n",
+    "- Containerization is useful when you need to ensure consistent behavior across various platforms, simplify deployment and scaling, and improve resource utilization.\n",
+    "- Common tools for deploying container-based architecture are Docker and Kubernetes."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "56890eec",
+   "metadata": {
+    "slideshow": {
+     "slide_type": "skip"
+    }
+   },
+   "source": [
+    "#### Serverless Architectures\n",
+    "- Serverless architectures are a cloud computing model where the cloud provider manages the infrastructure and automatically allocates resources based on the application's needs.\n",
+    "- Developers only need to focus on writing code, and the provider takes care of scaling, patching, and maintaining the underlying infrastructure. \n",
+    "- Serverless architectures can be useful when you want to reduce operational overhead, build event-driven applications, and optimize costs by paying only for the resources you actually use.\n",
+    "- Common tools to build serverless applications and APIs include Azure Functions, AWS Lambda, and Google Cloud Functions."
+   ]
   }
  ],
  "metadata": {
-  "celltoolbar": "Raw Cell Format",
+  "celltoolbar": "Slideshow",
   "kernelspec": {
    "display_name": "Python 3 (ipykernel)",
    "language": "python",