retrained after balancing dataset

train.ipynb

@@ -1,5 +1,54 @@
 {
   "cells": [
+    {
+      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Airline Sentiment Prediction using BERT\n",
+        "\n",
+        "### Approach\n",
+        "First I analysed the data and I found that there was a huge imbalance in the dataset, to resolve this I used Textattack for augumentation of data.\n",
+        "Before the augumenting the dataset I used the following techniques to clean the data & reduce the noise:\n",
+        "- Removed the @usernames\n",
+        "- Removed the URLs\n",
+        "- Removed hashtags\n",
+        "- Replacement of emojis with their meaning\n",
+        "\n",
+        "After cleaning the data I used EasyDataAugment of Textattack to augment the data, augmenting the data helped me to increase the accuracy of the model by more than 3%. I also tried using Clare(It replaces the words with their synonyms) but that was very resource intensive & it was taking very long to get output.\n",
+        "\n",
+        "### Model\n",
+        "Since, this was a binary classification task I used BERT for training the model. I used the pretrained BERT model from Huggingface transformers library. I used the BERT model with the following parameters:\n",
+        "- BERT-base-uncased\n",
+        "- Max length of the input sequence: 128\n",
+        "- Learning rate: 3e-5\n",
+        "- Batch size: 32\n",
+        "\n",
+        "### Results\n",
+        "The dataset was split into 80:20 ratio for training & validation.\n",
+        "I got the following results after training the model:\n",
+        "Training loss: 0.0137\n",
+        "Validation loss: 0.1209\n",
+        "Training accuracy: 0.9955\n",
+        "Validation accuracy: 0.9794\n"
+      ]
+    },
+    {
+      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "========================================================================================================================================"
+      ]
+    },
+    {
+      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "Install the required libraries"
+      ]
+    },
     {
       "cell_type": "code",
       "execution_count": null,
@@ -8,458 +57,79 @@
       },
       "outputs": [],
       "source": [
-        "%pip install \"tensorflow-gpu<2.11\"\n",
         "%pip install transformers\n",
         "%pip install emoji\n",
         "%pip install numpy pandas\n",
-        "%pip install scikit-learn"
+        "%pip install scikit-learn\n",
+        "%pip install textattack"
+      ]
+    },
+    {
+      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "Importing the libraries"
       ]
     },
     {
       "cell_type": "code",
-      "execution_count": 2,
+      "execution_count": null,
       "metadata": {
         "id": "iM2I9UEjm_pE"
       },
       "outputs": [],
       "source": [
         "import numpy as np\n",
-        "import pandas as pd"
+        "import pandas as pd\n",
+        "from pprint import pprint"
+      ]
+    },
+    {
+      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "Reading the data"
       ]
     },
     {
       "cell_type": "code",
-      "execution_count": 3,
+      "execution_count": null,
       "metadata": {
         "colab": {
           "base_uri": "https://localhost:8080/",
           "height": 676
         },
         "id": "mrnzcvkzm_pF",
-        "outputId": "fca854f8-422f-4868-e0c8-2c8abf4671f5"
+        "outputId": "61550835-27fc-4049-f3a3-f9ab9e1ba1bb"
       },
-      "outputs": [
-        {
-          "data": {
-            "text/html": [
-              "<div>\n",
-              "<style scoped>\n",
-              "    .dataframe tbody tr th:only-of-type {\n",
-              "        vertical-align: middle;\n",
-              "    }\n",
-              "\n",
-              "    .dataframe tbody tr th {\n",
-              "        vertical-align: top;\n",
-              "    }\n",
-              "\n",
-              "    .dataframe thead th {\n",
-              "        text-align: right;\n",
-              "    }\n",
-              "</style>\n",
-              "<table border=\"1\" class=\"dataframe\">\n",
-              "  <thead>\n",
-              "    <tr style=\"text-align: right;\">\n",
-              "      <th></th>\n",
-              "      <th>Unnamed: 0</th>\n",
-              "      <th>airline_sentiment</th>\n",
-              "      <th>text</th>\n",
-              "    </tr>\n",
-              "  </thead>\n",
-              "  <tbody>\n",
-              "    <tr>\n",
-              "      <th>0</th>\n",
-              "      <td>1</td>\n",
-              "      <td>positive</td>\n",
-              "      <td>@VirginAmerica plus you've added commercials t...</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>1</th>\n",
-              "      <td>3</td>\n",
-              "      <td>negative</td>\n",
-              "      <td>@VirginAmerica it's really aggressive to blast...</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>2</th>\n",
-              "      <td>4</td>\n",
-              "      <td>negative</td>\n",
-              "      <td>@VirginAmerica and it's a really big bad thing...</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>3</th>\n",
-              "      <td>5</td>\n",
-              "      <td>negative</td>\n",
-              "      <td>@VirginAmerica seriously would pay $30 a fligh...</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>4</th>\n",
-              "      <td>6</td>\n",
-              "      <td>positive</td>\n",
-              "      <td>@VirginAmerica yes, nearly every time I fly VX...</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>5</th>\n",
-              "      <td>8</td>\n",
-              "      <td>positive</td>\n",
-              "      <td>@virginamerica Well, I didn't…but NOW I DO! :-D</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>6</th>\n",
-              "      <td>9</td>\n",
-              "      <td>positive</td>\n",
-              "      <td>@VirginAmerica it was amazing, and arrived an ...</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>7</th>\n",
-              "      <td>11</td>\n",
-              "      <td>positive</td>\n",
-              "      <td>@VirginAmerica I &amp;lt;3 pretty graphics. so muc...</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>8</th>\n",
-              "      <td>12</td>\n",
-              "      <td>positive</td>\n",
-              "      <td>@VirginAmerica This is such a great deal! Alre...</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>9</th>\n",
-              "      <td>13</td>\n",
-              "      <td>positive</td>\n",
-              "      <td>@VirginAmerica @virginmedia I'm flying your #f...</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>10</th>\n",
-              "      <td>14</td>\n",
-              "      <td>positive</td>\n",
-              "      <td>@VirginAmerica Thanks!</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>11</th>\n",
-              "      <td>15</td>\n",
-              "      <td>negative</td>\n",
-              "      <td>@VirginAmerica SFO-PDX schedule is still MIA.</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>12</th>\n",
-              "      <td>16</td>\n",
-              "      <td>positive</td>\n",
-              "      <td>@VirginAmerica So excited for my first cross c...</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>13</th>\n",
-              "      <td>17</td>\n",
-              "      <td>negative</td>\n",
-              "      <td>@VirginAmerica  I flew from NYC to SFO last we...</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>14</th>\n",
-              "      <td>18</td>\n",
-              "      <td>positive</td>\n",
-              "      <td>I ❤️ flying @VirginAmerica. ☺️👍</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>15</th>\n",
-              "      <td>19</td>\n",
-              "      <td>positive</td>\n",
-              "      <td>@VirginAmerica you know what would be amazingl...</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>16</th>\n",
-              "      <td>20</td>\n",
-              "      <td>negative</td>\n",
-              "      <td>@VirginAmerica why are your first fares in May...</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>17</th>\n",
-              "      <td>21</td>\n",
-              "      <td>positive</td>\n",
-              "      <td>@VirginAmerica I love this graphic. http://t.c...</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>18</th>\n",
-              "      <td>22</td>\n",
-              "      <td>positive</td>\n",
-              "      <td>@VirginAmerica I love the hipster innovation. ...</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>19</th>\n",
-              "      <td>24</td>\n",
-              "      <td>negative</td>\n",
-              "      <td>@VirginAmerica you guys messed up my seating.....</td>\n",
-              "    </tr>\n",
-              "  </tbody>\n",
-              "</table>\n",
-              "</div>"
-            ],
-            "text/plain": [
-              "    Unnamed: 0 airline_sentiment  \\\n",
-              "0            1          positive   \n",
-              "1            3          negative   \n",
-              "2            4          negative   \n",
-              "3            5          negative   \n",
-              "4            6          positive   \n",
-              "5            8          positive   \n",
-              "6            9          positive   \n",
-              "7           11          positive   \n",
-              "8           12          positive   \n",
-              "9           13          positive   \n",
-              "10          14          positive   \n",
-              "11          15          negative   \n",
-              "12          16          positive   \n",
-              "13          17          negative   \n",
-              "14          18          positive   \n",
-              "15          19          positive   \n",
-              "16          20          negative   \n",
-              "17          21          positive   \n",
-              "18          22          positive   \n",
-              "19          24          negative   \n",
-              "\n",
-              "                                                 text  \n",
-              "0   @VirginAmerica plus you've added commercials t...  \n",
-              "1   @VirginAmerica it's really aggressive to blast...  \n",
-              "2   @VirginAmerica and it's a really big bad thing...  \n",
-              "3   @VirginAmerica seriously would pay $30 a fligh...  \n",
-              "4   @VirginAmerica yes, nearly every time I fly VX...  \n",
-              "5     @virginamerica Well, I didn't…but NOW I DO! :-D  \n",
-              "6   @VirginAmerica it was amazing, and arrived an ...  \n",
-              "7   @VirginAmerica I &lt;3 pretty graphics. so muc...  \n",
-              "8   @VirginAmerica This is such a great deal! Alre...  \n",
-              "9   @VirginAmerica @virginmedia I'm flying your #f...  \n",
-              "10                             @VirginAmerica Thanks!  \n",
-              "11      @VirginAmerica SFO-PDX schedule is still MIA.  \n",
-              "12  @VirginAmerica So excited for my first cross c...  \n",
-              "13  @VirginAmerica  I flew from NYC to SFO last we...  \n",
-              "14                    I ❤️ flying @VirginAmerica. ☺️👍  \n",
-              "15  @VirginAmerica you know what would be amazingl...  \n",
-              "16  @VirginAmerica why are your first fares in May...  \n",
-              "17  @VirginAmerica I love this graphic. http://t.c...  \n",
-              "18  @VirginAmerica I love the hipster innovation. ...  \n",
-              "19  @VirginAmerica you guys messed up my seating.....  "
-            ]
-          },
-          "execution_count": 3,
-          "metadata": {},
-          "output_type": "execute_result"
-        }
-      ],
+      "outputs": [],
       "source": [
         "df = pd.read_csv(\"airline_sentiment_analysis.csv\")\n",
-        "df.head(20)\n"
+        "df.head(20)"
+      ]
+    },
+    {
+      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "Assigning 1 to positive sentiment and 0 to negative sentiment"
       ]
     },
     {
       "cell_type": "code",
-      "execution_count": 4,
+      "execution_count": null,
       "metadata": {
         "colab": {
           "base_uri": "https://localhost:8080/",
           "height": 676
         },
         "id": "Jbl-wjpWm_pG",
-        "outputId": "0bf3597e-5645-4c37-a993-dc3c3eb57db4"
+        "outputId": "19c8b0ca-4506-4960-d588-505feecf678e"
       },
-      "outputs": [
-        {
-          "data": {
-            "text/html": [
-              "<div>\n",
-              "<style scoped>\n",
-              "    .dataframe tbody tr th:only-of-type {\n",
-              "        vertical-align: middle;\n",
-              "    }\n",
-              "\n",
-              "    .dataframe tbody tr th {\n",
-              "        vertical-align: top;\n",
-              "    }\n",
-              "\n",
-              "    .dataframe thead th {\n",
-              "        text-align: right;\n",
-              "    }\n",
-              "</style>\n",
-              "<table border=\"1\" class=\"dataframe\">\n",
-              "  <thead>\n",
-              "    <tr style=\"text-align: right;\">\n",
-              "      <th></th>\n",
-              "      <th>Unnamed: 0</th>\n",
-              "      <th>airline_sentiment</th>\n",
-              "      <th>text</th>\n",
-              "    </tr>\n",
-              "  </thead>\n",
-              "  <tbody>\n",
-              "    <tr>\n",
-              "      <th>0</th>\n",
-              "      <td>1</td>\n",
-              "      <td>1</td>\n",
-              "      <td>@VirginAmerica plus you've added commercials t...</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>1</th>\n",
-              "      <td>3</td>\n",
-              "      <td>0</td>\n",
-              "      <td>@VirginAmerica it's really aggressive to blast...</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>2</th>\n",
-              "      <td>4</td>\n",
-              "      <td>0</td>\n",
-              "      <td>@VirginAmerica and it's a really big bad thing...</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>3</th>\n",
-              "      <td>5</td>\n",
-              "      <td>0</td>\n",
-              "      <td>@VirginAmerica seriously would pay $30 a fligh...</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>4</th>\n",
-              "      <td>6</td>\n",
-              "      <td>1</td>\n",
-              "      <td>@VirginAmerica yes, nearly every time I fly VX...</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>5</th>\n",
-              "      <td>8</td>\n",
-              "      <td>1</td>\n",
-              "      <td>@virginamerica Well, I didn't…but NOW I DO! :-D</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>6</th>\n",
-              "      <td>9</td>\n",
-              "      <td>1</td>\n",
-              "      <td>@VirginAmerica it was amazing, and arrived an ...</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>7</th>\n",
-              "      <td>11</td>\n",
-              "      <td>1</td>\n",
-              "      <td>@VirginAmerica I &amp;lt;3 pretty graphics. so muc...</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>8</th>\n",
-              "      <td>12</td>\n",
-              "      <td>1</td>\n",
-              "      <td>@VirginAmerica This is such a great deal! Alre...</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>9</th>\n",
-              "      <td>13</td>\n",
-              "      <td>1</td>\n",
-              "      <td>@VirginAmerica @virginmedia I'm flying your #f...</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>10</th>\n",
-              "      <td>14</td>\n",
-              "      <td>1</td>\n",
-              "      <td>@VirginAmerica Thanks!</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>11</th>\n",
-              "      <td>15</td>\n",
-              "      <td>0</td>\n",
-              "      <td>@VirginAmerica SFO-PDX schedule is still MIA.</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>12</th>\n",
-              "      <td>16</td>\n",
-              "      <td>1</td>\n",
-              "      <td>@VirginAmerica So excited for my first cross c...</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>13</th>\n",
-              "      <td>17</td>\n",
-              "      <td>0</td>\n",
-              "      <td>@VirginAmerica  I flew from NYC to SFO last we...</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>14</th>\n",
-              "      <td>18</td>\n",
-              "      <td>1</td>\n",
-              "      <td>I ❤️ flying @VirginAmerica. ☺️👍</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>15</th>\n",
-              "      <td>19</td>\n",
-              "      <td>1</td>\n",
-              "      <td>@VirginAmerica you know what would be amazingl...</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>16</th>\n",
-              "      <td>20</td>\n",
-              "      <td>0</td>\n",
-              "      <td>@VirginAmerica why are your first fares in May...</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>17</th>\n",
-              "      <td>21</td>\n",
-              "      <td>1</td>\n",
-              "      <td>@VirginAmerica I love this graphic. http://t.c...</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>18</th>\n",
-              "      <td>22</td>\n",
-              "      <td>1</td>\n",
-              "      <td>@VirginAmerica I love the hipster innovation. ...</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>19</th>\n",
-              "      <td>24</td>\n",
-              "      <td>0</td>\n",
-              "      <td>@VirginAmerica you guys messed up my seating.....</td>\n",
-              "    </tr>\n",
-              "  </tbody>\n",
-              "</table>\n",
-              "</div>"
-            ],
-            "text/plain": [
-              "    Unnamed: 0  airline_sentiment  \\\n",
-              "0            1                  1   \n",
-              "1            3                  0   \n",
-              "2            4                  0   \n",
-              "3            5                  0   \n",
-              "4            6                  1   \n",
-              "5            8                  1   \n",
-              "6            9                  1   \n",
-              "7           11                  1   \n",
-              "8           12                  1   \n",
-              "9           13                  1   \n",
-              "10          14                  1   \n",
-              "11          15                  0   \n",
-              "12          16                  1   \n",
-              "13          17                  0   \n",
-              "14          18                  1   \n",
-              "15          19                  1   \n",
-              "16          20                  0   \n",
-              "17          21                  1   \n",
-              "18          22                  1   \n",
-              "19          24                  0   \n",
-              "\n",
-              "                                                 text  \n",
-              "0   @VirginAmerica plus you've added commercials t...  \n",
-              "1   @VirginAmerica it's really aggressive to blast...  \n",
-              "2   @VirginAmerica and it's a really big bad thing...  \n",
-              "3   @VirginAmerica seriously would pay $30 a fligh...  \n",
-              "4   @VirginAmerica yes, nearly every time I fly VX...  \n",
-              "5     @virginamerica Well, I didn't…but NOW I DO! :-D  \n",
-              "6   @VirginAmerica it was amazing, and arrived an ...  \n",
-              "7   @VirginAmerica I &lt;3 pretty graphics. so muc...  \n",
-              "8   @VirginAmerica This is such a great deal! Alre...  \n",
-              "9   @VirginAmerica @virginmedia I'm flying your #f...  \n",
-              "10                             @VirginAmerica Thanks!  \n",
-              "11      @VirginAmerica SFO-PDX schedule is still MIA.  \n",
-              "12  @VirginAmerica So excited for my first cross c...  \n",
-              "13  @VirginAmerica  I flew from NYC to SFO last we...  \n",
-              "14                    I ❤️ flying @VirginAmerica. ☺️👍  \n",
-              "15  @VirginAmerica you know what would be amazingl...  \n",
-              "16  @VirginAmerica why are your first fares in May...  \n",
-              "17  @VirginAmerica I love this graphic. http://t.c...  \n",
-              "18  @VirginAmerica I love the hipster innovation. ...  \n",
-              "19  @VirginAmerica you guys messed up my seating.....  "
-            ]
-          },
-          "execution_count": 4,
-          "metadata": {},
-          "output_type": "execute_result"
-        }
-      ],
+      "outputs": [],
       "source": [
         "for label in df['airline_sentiment']:\n",
         "    if label == 'positive':\n",
@@ -469,223 +139,28 @@
         "df.head(20)"
       ]
     },
+    {
+      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "Remove the @usernames, URLs, hashtags & Replace the emojis with their meaning"
+      ]
+    },
     {
       "cell_type": "code",
-      "execution_count": 5,
+      "execution_count": null,
       "metadata": {
         "colab": {
           "base_uri": "https://localhost:8080/",
           "height": 676
         },
         "id": "ApZHkGw2m_pG",
-        "outputId": "3b89a43e-9523-42be-d646-679858136abc"
+        "outputId": "330f44ff-3314-4392-eef4-69f2994b1cae"
       },
-      "outputs": [
-        {
-          "data": {
-            "text/html": [
-              "<div>\n",
-              "<style scoped>\n",
-              "    .dataframe tbody tr th:only-of-type {\n",
-              "        vertical-align: middle;\n",
-              "    }\n",
-              "\n",
-              "    .dataframe tbody tr th {\n",
-              "        vertical-align: top;\n",
-              "    }\n",
-              "\n",
-              "    .dataframe thead th {\n",
-              "        text-align: right;\n",
-              "    }\n",
-              "</style>\n",
-              "<table border=\"1\" class=\"dataframe\">\n",
-              "  <thead>\n",
-              "    <tr style=\"text-align: right;\">\n",
-              "      <th></th>\n",
-              "      <th>Unnamed: 0</th>\n",
-              "      <th>airline_sentiment</th>\n",
-              "      <th>text</th>\n",
-              "    </tr>\n",
-              "  </thead>\n",
-              "  <tbody>\n",
-              "    <tr>\n",
-              "      <th>0</th>\n",
-              "      <td>1</td>\n",
-              "      <td>1</td>\n",
-              "      <td>plus you've added commercials to the experien...</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>1</th>\n",
-              "      <td>3</td>\n",
-              "      <td>0</td>\n",
-              "      <td>it's really aggressive to blast obnoxious  en...</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>2</th>\n",
-              "      <td>4</td>\n",
-              "      <td>0</td>\n",
-              "      <td>and it's a really big bad thing about it</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>3</th>\n",
-              "      <td>5</td>\n",
-              "      <td>0</td>\n",
-              "      <td>seriously would pay     a flight for seats th...</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>4</th>\n",
-              "      <td>6</td>\n",
-              "      <td>1</td>\n",
-              "      <td>yes  nearly every time I fly VX this  ear wor...</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>5</th>\n",
-              "      <td>8</td>\n",
-              "      <td>1</td>\n",
-              "      <td>Well  I didn't but NOW I DO!  D</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>6</th>\n",
-              "      <td>9</td>\n",
-              "      <td>1</td>\n",
-              "      <td>it was amazing  and arrived an hour early. Yo...</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>7</th>\n",
-              "      <td>11</td>\n",
-              "      <td>1</td>\n",
-              "      <td>I  lt   pretty graphics. so much better than ...</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>8</th>\n",
-              "      <td>12</td>\n",
-              "      <td>1</td>\n",
-              "      <td>This is such a great deal! Already thinking a...</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>9</th>\n",
-              "      <td>13</td>\n",
-              "      <td>1</td>\n",
-              "      <td>I'm flying your fabulous Seductive skies aga...</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>10</th>\n",
-              "      <td>14</td>\n",
-              "      <td>1</td>\n",
-              "      <td>Thanks!</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>11</th>\n",
-              "      <td>15</td>\n",
-              "      <td>0</td>\n",
-              "      <td>SFO PDX schedule is still MIA.</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>12</th>\n",
-              "      <td>16</td>\n",
-              "      <td>1</td>\n",
-              "      <td>So excited for my first cross country flight ...</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>13</th>\n",
-              "      <td>17</td>\n",
-              "      <td>0</td>\n",
-              "      <td>I flew from NYC to SFO last week and couldn't...</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>14</th>\n",
-              "      <td>18</td>\n",
-              "      <td>1</td>\n",
-              "      <td>I red heart flying . smiling face thumbs up</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>15</th>\n",
-              "      <td>19</td>\n",
-              "      <td>1</td>\n",
-              "      <td>you know what would be amazingly awesome? BOS...</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>16</th>\n",
-              "      <td>20</td>\n",
-              "      <td>0</td>\n",
-              "      <td>why are your first fares in May over three ti...</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>17</th>\n",
-              "      <td>21</td>\n",
-              "      <td>1</td>\n",
-              "      <td>I love this graphic. http   t.co UT GrRwAaA</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>18</th>\n",
-              "      <td>22</td>\n",
-              "      <td>1</td>\n",
-              "      <td>I love the hipster innovation. You are a feel...</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>19</th>\n",
-              "      <td>24</td>\n",
-              "      <td>0</td>\n",
-              "      <td>you guys messed up my seating.. I reserved se...</td>\n",
-              "    </tr>\n",
-              "  </tbody>\n",
-              "</table>\n",
-              "</div>"
-            ],
-            "text/plain": [
-              "    Unnamed: 0  airline_sentiment  \\\n",
-              "0            1                  1   \n",
-              "1            3                  0   \n",
-              "2            4                  0   \n",
-              "3            5                  0   \n",
-              "4            6                  1   \n",
-              "5            8                  1   \n",
-              "6            9                  1   \n",
-              "7           11                  1   \n",
-              "8           12                  1   \n",
-              "9           13                  1   \n",
-              "10          14                  1   \n",
-              "11          15                  0   \n",
-              "12          16                  1   \n",
-              "13          17                  0   \n",
-              "14          18                  1   \n",
-              "15          19                  1   \n",
-              "16          20                  0   \n",
-              "17          21                  1   \n",
-              "18          22                  1   \n",
-              "19          24                  0   \n",
-              "\n",
-              "                                                 text  \n",
-              "0    plus you've added commercials to the experien...  \n",
-              "1    it's really aggressive to blast obnoxious  en...  \n",
-              "2            and it's a really big bad thing about it  \n",
-              "3    seriously would pay     a flight for seats th...  \n",
-              "4    yes  nearly every time I fly VX this  ear wor...  \n",
-              "5                     Well  I didn't but NOW I DO!  D  \n",
-              "6    it was amazing  and arrived an hour early. Yo...  \n",
-              "7    I  lt   pretty graphics. so much better than ...  \n",
-              "8    This is such a great deal! Already thinking a...  \n",
-              "9     I'm flying your fabulous Seductive skies aga...  \n",
-              "10                                            Thanks!  \n",
-              "11                     SFO PDX schedule is still MIA.  \n",
-              "12   So excited for my first cross country flight ...  \n",
-              "13   I flew from NYC to SFO last week and couldn't...  \n",
-              "14        I red heart flying . smiling face thumbs up  \n",
-              "15   you know what would be amazingly awesome? BOS...  \n",
-              "16   why are your first fares in May over three ti...  \n",
-              "17        I love this graphic. http   t.co UT GrRwAaA  \n",
-              "18   I love the hipster innovation. You are a feel...  \n",
-              "19   you guys messed up my seating.. I reserved se...  "
-            ]
-          },
-          "execution_count": 5,
-          "metadata": {},
-          "output_type": "execute_result"
-        }
-      ],
+      "outputs": [],
       "source": [
-        "# remove mentions, hashtags, links, and special characters\n",
-        "# replace emoji with text\n",
+        "\n",
         "import emoji\n",
         "for i,r in df.iterrows():\n",
         "  \n",
@@ -702,9 +177,56 @@
         "df.head(20)"
       ]
     },
+    {
+      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "Augumenting Positive Sentiment using EasyDataAugment"
+      ]
+    },
     {
       "cell_type": "code",
-      "execution_count": 8,
+      "execution_count": null,
+      "metadata": {
+        "id": "lNr4OWDaGYww"
+      },
+      "outputs": [],
+      "source": [
+        "positive_feedback = (df.loc[df[\"airline_sentiment\"] == 1])[\"text\"]\n",
+        "positive_feedback = positive_feedback.tolist()\n",
+        "# pprint(positive_feedback)\n",
+        "\n",
+        "from textattack.augmentation import EasyDataAugmenter\n",
+        "esy_aug = EasyDataAugmenter()\n",
+        "aug_list = []\n",
+        "for sen in positive_feedback:\n",
+        "  aug_list.append(esy_aug.augment(sen))\n",
+        "serial_list = []\n",
+        "for l in aug_list:\n",
+        "  for sen in l:\n",
+        "    serial_list.append(sen)\n",
+        "df = df.drop(df.columns[[0]],axis=1)\n",
+        "\n",
+        "df2 = pd.DataFrame(list(zip([1]*len(serial_list),serial_list)),columns=[\"airline_sentiment\",\"text\"])\n",
+        "\n",
+        "df = pd.concat([df,df2])\n",
+        "\n",
+        "df.to_csv(\"modified.csv\") #save the modified dataset\n",
+        "df.head()"
+      ]
+    },
+    {
+      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "Split dataset into train & validation in 80:20 ratio"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
       "metadata": {
         "id": "MrynUQ9Xm_pG"
       },
@@ -716,9 +238,17 @@
         "train, test = train_test_split(df, test_size=0.2, random_state=42)\n"
       ]
     },
+    {
+      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "Initalise the BERT model & tokenizer"
+      ]
+    },
     {
       "cell_type": "code",
-      "execution_count": 9,
+      "execution_count": null,
       "metadata": {
         "colab": {
           "base_uri": "https://localhost:8080/"
@@ -726,20 +256,7 @@
         "id": "MambfTNXm_pG",
         "outputId": "f0e11223-8e74-445a-8cdc-cc8492f26b14"
       },
-      "outputs": [
-        {
-          "name": "stderr",
-          "output_type": "stream",
-          "text": [
-            "d:\\sentiment_analysis\\venv\\lib\\site-packages\\tqdm\\auto.py:22: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n",
-            "  from .autonotebook import tqdm as notebook_tqdm\n",
-            "All model checkpoint layers were used when initializing TFBertForSequenceClassification.\n",
-            "\n",
-            "Some layers of TFBertForSequenceClassification were not initialized from the model checkpoint at bert-base-uncased and are newly initialized: ['classifier']\n",
-            "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
-          ]
-        }
-      ],
+      "outputs": [],
       "source": [
         "from transformers import BertTokenizer, TFBertForSequenceClassification\n",
         "from transformers import InputExample, InputFeatures\n",
@@ -749,9 +266,17 @@
         "tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')"
       ]
     },
+    {
+      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "Utility function to convert the data into the format required by BERT"
+      ]
+    },
     {
       "cell_type": "code",
-      "execution_count": 10,
+      "execution_count": null,
       "metadata": {
         "id": "uxgZ7GsEm_pH"
       },
@@ -770,10 +295,6 @@
         "  \n",
         "  return train_InputExamples, validation_InputExamples\n",
         "\n",
-        "  train_InputExamples, validation_InputExamples = convert_data_to_examples(train, \n",
-        "                                                                           test, \n",
-        "                                                                           'DATA_COLUMN', \n",
-        "                                                                           'LABEL_COLUMN')\n",
         "  \n",
         "def convert_examples_to_tf_dataset(examples, tokenizer, max_length=128):\n",
         "    features = [] # -> will hold InputFeatures to be converted later\n",
@@ -824,9 +345,17 @@
         "    )\n"
       ]
     },
+    {
+      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "BERT model for training"
+      ]
+    },
     {
       "cell_type": "code",
-      "execution_count": 11,
+      "execution_count": null,
       "metadata": {
         "colab": {
           "base_uri": "https://localhost:8080/"
@@ -834,16 +363,7 @@
         "id": "tPsHpWhJm_pH",
         "outputId": "a9f7b2b8-d0bb-474b-d91a-25f0c8a40905"
       },
-      "outputs": [
-        {
-          "name": "stderr",
-          "output_type": "stream",
-          "text": [
-            "d:\\sentiment_analysis\\venv\\lib\\site-packages\\transformers\\tokenization_utils_base.py:2336: FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in a future version, use `padding=True` or `padding='longest'` to pad to the longest sequence in the batch, or use `padding='max_length'` to pad to a max length. In this case, you can give a specific length with `max_length` (e.g. `max_length=45`) or leave max_length to None to pad to the maximal input size of the model (e.g. 512 for Bert).\n",
-            "  warnings.warn(\n"
-          ]
-        }
-      ],
+      "outputs": [],
       "source": [
         "DATA_COLUMN = 'text'\n",
         "LABEL_COLUMN = 'airline_sentiment'\n",
@@ -860,7 +380,7 @@
     },
     {
       "cell_type": "code",
-      "execution_count": 13,
+      "execution_count": null,
       "metadata": {
         "colab": {
           "base_uri": "https://localhost:8080/"
@@ -868,36 +388,7 @@
         "id": "GDcgmUOCm_pI",
         "outputId": "2f262b78-65f4-4cfc-deb3-a51d2b499eab"
       },
-      "outputs": [
-        {
-          "name": "stdout",
-          "output_type": "stream",
-          "text": [
-            "Epoch 1/2\n",
-            "      4/Unknown - 168s 36s/step - loss: 0.4485 - accuracy: 0.8203"
-          ]
-        },
-        {
-          "ename": "KeyboardInterrupt",
-          "evalue": "",
-          "output_type": "error",
-          "traceback": [
-            "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
-            "\u001b[1;31mKeyboardInterrupt\u001b[0m                         Traceback (most recent call last)",
-            "Cell \u001b[1;32mIn[13], line 5\u001b[0m\n\u001b[0;32m      1\u001b[0m model\u001b[39m.\u001b[39mcompile(optimizer\u001b[39m=\u001b[39mtf\u001b[39m.\u001b[39mkeras\u001b[39m.\u001b[39moptimizers\u001b[39m.\u001b[39mAdam(learning_rate\u001b[39m=\u001b[39m\u001b[39m3e-5\u001b[39m, epsilon\u001b[39m=\u001b[39m\u001b[39m1e-08\u001b[39m, clipnorm\u001b[39m=\u001b[39m\u001b[39m1.0\u001b[39m), \n\u001b[0;32m      2\u001b[0m               loss\u001b[39m=\u001b[39mtf\u001b[39m.\u001b[39mkeras\u001b[39m.\u001b[39mlosses\u001b[39m.\u001b[39mSparseCategoricalCrossentropy(from_logits\u001b[39m=\u001b[39m\u001b[39mTrue\u001b[39;00m), \n\u001b[0;32m      3\u001b[0m               metrics\u001b[39m=\u001b[39m[tf\u001b[39m.\u001b[39mkeras\u001b[39m.\u001b[39mmetrics\u001b[39m.\u001b[39mSparseCategoricalAccuracy(\u001b[39m'\u001b[39m\u001b[39maccuracy\u001b[39m\u001b[39m'\u001b[39m)])\n\u001b[1;32m----> 5\u001b[0m model\u001b[39m.\u001b[39;49mfit(train_data, epochs\u001b[39m=\u001b[39;49m\u001b[39m2\u001b[39;49m, validation_data\u001b[39m=\u001b[39;49mvalidation_data)\n",
-            "File \u001b[1;32md:\\sentiment_analysis\\venv\\lib\\site-packages\\keras\\utils\\traceback_utils.py:65\u001b[0m, in \u001b[0;36mfilter_traceback.<locals>.error_handler\u001b[1;34m(*args, **kwargs)\u001b[0m\n\u001b[0;32m     63\u001b[0m filtered_tb \u001b[39m=\u001b[39m \u001b[39mNone\u001b[39;00m\n\u001b[0;32m     64\u001b[0m \u001b[39mtry\u001b[39;00m:\n\u001b[1;32m---> 65\u001b[0m     \u001b[39mreturn\u001b[39;00m fn(\u001b[39m*\u001b[39margs, \u001b[39m*\u001b[39m\u001b[39m*\u001b[39mkwargs)\n\u001b[0;32m     66\u001b[0m \u001b[39mexcept\u001b[39;00m \u001b[39mException\u001b[39;00m \u001b[39mas\u001b[39;00m e:\n\u001b[0;32m     67\u001b[0m     filtered_tb \u001b[39m=\u001b[39m _process_traceback_frames(e\u001b[39m.\u001b[39m__traceback__)\n",
-            "File \u001b[1;32md:\\sentiment_analysis\\venv\\lib\\site-packages\\keras\\engine\\training.py:1564\u001b[0m, in \u001b[0;36mModel.fit\u001b[1;34m(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_batch_size, validation_freq, max_queue_size, workers, use_multiprocessing)\u001b[0m\n\u001b[0;32m   1556\u001b[0m \u001b[39mwith\u001b[39;00m tf\u001b[39m.\u001b[39mprofiler\u001b[39m.\u001b[39mexperimental\u001b[39m.\u001b[39mTrace(\n\u001b[0;32m   1557\u001b[0m     \u001b[39m\"\u001b[39m\u001b[39mtrain\u001b[39m\u001b[39m\"\u001b[39m,\n\u001b[0;32m   1558\u001b[0m     epoch_num\u001b[39m=\u001b[39mepoch,\n\u001b[1;32m   (...)\u001b[0m\n\u001b[0;32m   1561\u001b[0m     _r\u001b[39m=\u001b[39m\u001b[39m1\u001b[39m,\n\u001b[0;32m   1562\u001b[0m ):\n\u001b[0;32m   1563\u001b[0m     callbacks\u001b[39m.\u001b[39mon_train_batch_begin(step)\n\u001b[1;32m-> 1564\u001b[0m     tmp_logs \u001b[39m=\u001b[39m \u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49mtrain_function(iterator)\n\u001b[0;32m   1565\u001b[0m     \u001b[39mif\u001b[39;00m data_handler\u001b[39m.\u001b[39mshould_sync:\n\u001b[0;32m   1566\u001b[0m         context\u001b[39m.\u001b[39masync_wait()\n",
-            "File \u001b[1;32md:\\sentiment_analysis\\venv\\lib\\site-packages\\tensorflow\\python\\util\\traceback_utils.py:150\u001b[0m, in \u001b[0;36mfilter_traceback.<locals>.error_handler\u001b[1;34m(*args, **kwargs)\u001b[0m\n\u001b[0;32m    148\u001b[0m filtered_tb \u001b[39m=\u001b[39m \u001b[39mNone\u001b[39;00m\n\u001b[0;32m    149\u001b[0m \u001b[39mtry\u001b[39;00m:\n\u001b[1;32m--> 150\u001b[0m   \u001b[39mreturn\u001b[39;00m fn(\u001b[39m*\u001b[39margs, \u001b[39m*\u001b[39m\u001b[39m*\u001b[39mkwargs)\n\u001b[0;32m    151\u001b[0m \u001b[39mexcept\u001b[39;00m \u001b[39mException\u001b[39;00m \u001b[39mas\u001b[39;00m e:\n\u001b[0;32m    152\u001b[0m   filtered_tb \u001b[39m=\u001b[39m _process_traceback_frames(e\u001b[39m.\u001b[39m__traceback__)\n",
-            "File \u001b[1;32md:\\sentiment_analysis\\venv\\lib\\site-packages\\tensorflow\\python\\eager\\def_function.py:915\u001b[0m, in \u001b[0;36mFunction.__call__\u001b[1;34m(self, *args, **kwds)\u001b[0m\n\u001b[0;32m    912\u001b[0m compiler \u001b[39m=\u001b[39m \u001b[39m\"\u001b[39m\u001b[39mxla\u001b[39m\u001b[39m\"\u001b[39m \u001b[39mif\u001b[39;00m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_jit_compile \u001b[39melse\u001b[39;00m \u001b[39m\"\u001b[39m\u001b[39mnonXla\u001b[39m\u001b[39m\"\u001b[39m\n\u001b[0;32m    914\u001b[0m \u001b[39mwith\u001b[39;00m OptionalXlaContext(\u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_jit_compile):\n\u001b[1;32m--> 915\u001b[0m   result \u001b[39m=\u001b[39m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_call(\u001b[39m*\u001b[39margs, \u001b[39m*\u001b[39m\u001b[39m*\u001b[39mkwds)\n\u001b[0;32m    917\u001b[0m new_tracing_count \u001b[39m=\u001b[39m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39mexperimental_get_tracing_count()\n\u001b[0;32m    918\u001b[0m without_tracing \u001b[39m=\u001b[39m (tracing_count \u001b[39m==\u001b[39m new_tracing_count)\n",
-            "File \u001b[1;32md:\\sentiment_analysis\\venv\\lib\\site-packages\\tensorflow\\python\\eager\\def_function.py:947\u001b[0m, in \u001b[0;36mFunction._call\u001b[1;34m(self, *args, **kwds)\u001b[0m\n\u001b[0;32m    944\u001b[0m   \u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_lock\u001b[39m.\u001b[39mrelease()\n\u001b[0;32m    945\u001b[0m   \u001b[39m# In this case we have created variables on the first call, so we run the\u001b[39;00m\n\u001b[0;32m    946\u001b[0m   \u001b[39m# defunned version which is guaranteed to never create variables.\u001b[39;00m\n\u001b[1;32m--> 947\u001b[0m   \u001b[39mreturn\u001b[39;00m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_stateless_fn(\u001b[39m*\u001b[39margs, \u001b[39m*\u001b[39m\u001b[39m*\u001b[39mkwds)  \u001b[39m# pylint: disable=not-callable\u001b[39;00m\n\u001b[0;32m    948\u001b[0m \u001b[39melif\u001b[39;00m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_stateful_fn \u001b[39mis\u001b[39;00m \u001b[39mnot\u001b[39;00m \u001b[39mNone\u001b[39;00m:\n\u001b[0;32m    949\u001b[0m   \u001b[39m# Release the lock early so that multiple threads can perform the call\u001b[39;00m\n\u001b[0;32m    950\u001b[0m   \u001b[39m# in parallel.\u001b[39;00m\n\u001b[0;32m    951\u001b[0m   \u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_lock\u001b[39m.\u001b[39mrelease()\n",
-            "File \u001b[1;32md:\\sentiment_analysis\\venv\\lib\\site-packages\\tensorflow\\python\\eager\\function.py:2496\u001b[0m, in \u001b[0;36mFunction.__call__\u001b[1;34m(self, *args, **kwargs)\u001b[0m\n\u001b[0;32m   2493\u001b[0m \u001b[39mwith\u001b[39;00m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_lock:\n\u001b[0;32m   2494\u001b[0m   (graph_function,\n\u001b[0;32m   2495\u001b[0m    filtered_flat_args) \u001b[39m=\u001b[39m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_maybe_define_function(args, kwargs)\n\u001b[1;32m-> 2496\u001b[0m \u001b[39mreturn\u001b[39;00m graph_function\u001b[39m.\u001b[39;49m_call_flat(\n\u001b[0;32m   2497\u001b[0m     filtered_flat_args, captured_inputs\u001b[39m=\u001b[39;49mgraph_function\u001b[39m.\u001b[39;49mcaptured_inputs)\n",
-            "File \u001b[1;32md:\\sentiment_analysis\\venv\\lib\\site-packages\\tensorflow\\python\\eager\\function.py:1862\u001b[0m, in \u001b[0;36mConcreteFunction._call_flat\u001b[1;34m(self, args, captured_inputs, cancellation_manager)\u001b[0m\n\u001b[0;32m   1858\u001b[0m possible_gradient_type \u001b[39m=\u001b[39m gradients_util\u001b[39m.\u001b[39mPossibleTapeGradientTypes(args)\n\u001b[0;32m   1859\u001b[0m \u001b[39mif\u001b[39;00m (possible_gradient_type \u001b[39m==\u001b[39m gradients_util\u001b[39m.\u001b[39mPOSSIBLE_GRADIENT_TYPES_NONE\n\u001b[0;32m   1860\u001b[0m     \u001b[39mand\u001b[39;00m executing_eagerly):\n\u001b[0;32m   1861\u001b[0m   \u001b[39m# No tape is watching; skip to running the function.\u001b[39;00m\n\u001b[1;32m-> 1862\u001b[0m   \u001b[39mreturn\u001b[39;00m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_build_call_outputs(\u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49m_inference_function\u001b[39m.\u001b[39;49mcall(\n\u001b[0;32m   1863\u001b[0m       ctx, args, cancellation_manager\u001b[39m=\u001b[39;49mcancellation_manager))\n\u001b[0;32m   1864\u001b[0m forward_backward \u001b[39m=\u001b[39m \u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_select_forward_and_backward_functions(\n\u001b[0;32m   1865\u001b[0m     args,\n\u001b[0;32m   1866\u001b[0m     possible_gradient_type,\n\u001b[0;32m   1867\u001b[0m     executing_eagerly)\n\u001b[0;32m   1868\u001b[0m forward_function, args_with_tangents \u001b[39m=\u001b[39m forward_backward\u001b[39m.\u001b[39mforward()\n",
-            "File \u001b[1;32md:\\sentiment_analysis\\venv\\lib\\site-packages\\tensorflow\\python\\eager\\function.py:499\u001b[0m, in \u001b[0;36m_EagerDefinedFunction.call\u001b[1;34m(self, ctx, args, cancellation_manager)\u001b[0m\n\u001b[0;32m    497\u001b[0m \u001b[39mwith\u001b[39;00m _InterpolateFunctionError(\u001b[39mself\u001b[39m):\n\u001b[0;32m    498\u001b[0m   \u001b[39mif\u001b[39;00m cancellation_manager \u001b[39mis\u001b[39;00m \u001b[39mNone\u001b[39;00m:\n\u001b[1;32m--> 499\u001b[0m     outputs \u001b[39m=\u001b[39m execute\u001b[39m.\u001b[39;49mexecute(\n\u001b[0;32m    500\u001b[0m         \u001b[39mstr\u001b[39;49m(\u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49msignature\u001b[39m.\u001b[39;49mname),\n\u001b[0;32m    501\u001b[0m         num_outputs\u001b[39m=\u001b[39;49m\u001b[39mself\u001b[39;49m\u001b[39m.\u001b[39;49m_num_outputs,\n\u001b[0;32m    502\u001b[0m         inputs\u001b[39m=\u001b[39;49margs,\n\u001b[0;32m    503\u001b[0m         attrs\u001b[39m=\u001b[39;49mattrs,\n\u001b[0;32m    504\u001b[0m         ctx\u001b[39m=\u001b[39;49mctx)\n\u001b[0;32m    505\u001b[0m   \u001b[39melse\u001b[39;00m:\n\u001b[0;32m    506\u001b[0m     outputs \u001b[39m=\u001b[39m execute\u001b[39m.\u001b[39mexecute_with_cancellation(\n\u001b[0;32m    507\u001b[0m         \u001b[39mstr\u001b[39m(\u001b[39mself\u001b[39m\u001b[39m.\u001b[39msignature\u001b[39m.\u001b[39mname),\n\u001b[0;32m    508\u001b[0m         num_outputs\u001b[39m=\u001b[39m\u001b[39mself\u001b[39m\u001b[39m.\u001b[39m_num_outputs,\n\u001b[1;32m   (...)\u001b[0m\n\u001b[0;32m    511\u001b[0m         ctx\u001b[39m=\u001b[39mctx,\n\u001b[0;32m    512\u001b[0m         cancellation_manager\u001b[39m=\u001b[39mcancellation_manager)\n",
-            "File \u001b[1;32md:\\sentiment_analysis\\venv\\lib\\site-packages\\tensorflow\\python\\eager\\execute.py:54\u001b[0m, in \u001b[0;36mquick_execute\u001b[1;34m(op_name, num_outputs, inputs, attrs, ctx, name)\u001b[0m\n\u001b[0;32m     52\u001b[0m \u001b[39mtry\u001b[39;00m:\n\u001b[0;32m     53\u001b[0m   ctx\u001b[39m.\u001b[39mensure_initialized()\n\u001b[1;32m---> 54\u001b[0m   tensors \u001b[39m=\u001b[39m pywrap_tfe\u001b[39m.\u001b[39;49mTFE_Py_Execute(ctx\u001b[39m.\u001b[39;49m_handle, device_name, op_name,\n\u001b[0;32m     55\u001b[0m                                       inputs, attrs, num_outputs)\n\u001b[0;32m     56\u001b[0m \u001b[39mexcept\u001b[39;00m core\u001b[39m.\u001b[39m_NotOkStatusException \u001b[39mas\u001b[39;00m e:\n\u001b[0;32m     57\u001b[0m   \u001b[39mif\u001b[39;00m name \u001b[39mis\u001b[39;00m \u001b[39mnot\u001b[39;00m \u001b[39mNone\u001b[39;00m:\n",
-            "\u001b[1;31mKeyboardInterrupt\u001b[0m: "
-          ]
-        }
-      ],
+      "outputs": [],
       "source": [
         "model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=3e-5, epsilon=1e-08, clipnorm=1.0), \n",
         "              loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True), \n",
@@ -906,6 +397,14 @@
         "model.fit(train_data, epochs=2, validation_data=validation_data)"
       ]
     },
+    {
+      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "Saving the trained weights"
+      ]
+    },
     {
       "cell_type": "code",
       "execution_count": null,
@@ -917,6 +416,14 @@
         "model.save_weights(\"weights.h5\")"
       ]
     },
+    {
+      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "Inference: Predicting the sentiment of the tweet"
+      ]
+    },
     {
       "cell_type": "code",
       "execution_count": null,
@@ -928,139 +435,7 @@
         "id": "BLcz_yKOr38C",
         "outputId": "a56245b6-395a-4ff1-aeba-ab30c98cedb1"
       },
-      "outputs": [
-        {
-          "data": {
-            "text/html": [
-              "\n",
-              "  <div id=\"df-cc5f72e8-1526-491c-af16-8e253c1ba29b\">\n",
-              "    <div class=\"colab-df-container\">\n",
-              "      <div>\n",
-              "<style scoped>\n",
-              "    .dataframe tbody tr th:only-of-type {\n",
-              "        vertical-align: middle;\n",
-              "    }\n",
-              "\n",
-              "    .dataframe tbody tr th {\n",
-              "        vertical-align: top;\n",
-              "    }\n",
-              "\n",
-              "    .dataframe thead th {\n",
-              "        text-align: right;\n",
-              "    }\n",
-              "</style>\n",
-              "<table border=\"1\" class=\"dataframe\">\n",
-              "  <thead>\n",
-              "    <tr style=\"text-align: right;\">\n",
-              "      <th></th>\n",
-              "      <th>0</th>\n",
-              "    </tr>\n",
-              "  </thead>\n",
-              "  <tbody>\n",
-              "    <tr>\n",
-              "      <th>0</th>\n",
-              "      <td>The flight was great</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>1</th>\n",
-              "      <td>frowning face</td>\n",
-              "    </tr>\n",
-              "    <tr>\n",
-              "      <th>2</th>\n",
-              "      <td>confetti ball it was bad experience</td>\n",
-              "    </tr>\n",
-              "  </tbody>\n",
-              "</table>\n",
-              "</div>\n",
-              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-cc5f72e8-1526-491c-af16-8e253c1ba29b')\"\n",
-              "              title=\"Convert this dataframe to an interactive table.\"\n",
-              "              style=\"display:none;\">\n",
-              "        \n",
-              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
-              "       width=\"24px\">\n",
-              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
-              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
-              "  </svg>\n",
-              "      </button>\n",
-              "      \n",
-              "  <style>\n",
-              "    .colab-df-container {\n",
-              "      display:flex;\n",
-              "      flex-wrap:wrap;\n",
-              "      gap: 12px;\n",
-              "    }\n",
-              "\n",
-              "    .colab-df-convert {\n",
-              "      background-color: #E8F0FE;\n",
-              "      border: none;\n",
-              "      border-radius: 50%;\n",
-              "      cursor: pointer;\n",
-              "      display: none;\n",
-              "      fill: #1967D2;\n",
-              "      height: 32px;\n",
-              "      padding: 0 0 0 0;\n",
-              "      width: 32px;\n",
-              "    }\n",
-              "\n",
-              "    .colab-df-convert:hover {\n",
-              "      background-color: #E2EBFA;\n",
-              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
-              "      fill: #174EA6;\n",
-              "    }\n",
-              "\n",
-              "    [theme=dark] .colab-df-convert {\n",
-              "      background-color: #3B4455;\n",
-              "      fill: #D2E3FC;\n",
-              "    }\n",
-              "\n",
-              "    [theme=dark] .colab-df-convert:hover {\n",
-              "      background-color: #434B5C;\n",
-              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
-              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
-              "      fill: #FFFFFF;\n",
-              "    }\n",
-              "  </style>\n",
-              "\n",
-              "      <script>\n",
-              "        const buttonEl =\n",
-              "          document.querySelector('#df-cc5f72e8-1526-491c-af16-8e253c1ba29b button.colab-df-convert');\n",
-              "        buttonEl.style.display =\n",
-              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
-              "\n",
-              "        async function convertToInteractive(key) {\n",
-              "          const element = document.querySelector('#df-cc5f72e8-1526-491c-af16-8e253c1ba29b');\n",
-              "          const dataTable =\n",
-              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
-              "                                                     [key], {});\n",
-              "          if (!dataTable) return;\n",
-              "\n",
-              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
-              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
-              "            + ' to learn more about interactive tables.';\n",
-              "          element.innerHTML = '';\n",
-              "          dataTable['output_type'] = 'display_data';\n",
-              "          await google.colab.output.renderOutput(dataTable, element);\n",
-              "          const docLink = document.createElement('div');\n",
-              "          docLink.innerHTML = docLinkHtml;\n",
-              "          element.appendChild(docLink);\n",
-              "        }\n",
-              "      </script>\n",
-              "    </div>\n",
-              "  </div>\n",
-              "  "
-            ],
-            "text/plain": [
-              "                                     0\n",
-              "0                 The flight was great\n",
-              "1                        frowning face\n",
-              "2  confetti ball it was bad experience"
-            ]
-          },
-          "execution_count": 23,
-          "metadata": {},
-          "output_type": "execute_result"
-        }
-      ],
+      "outputs": [],
       "source": [
         "pred_data = [\"@abc The flight was great\", \"@abc ☹️\",\"🎊 it was bad experience\"]\n",
         "pred_data = pd.DataFrame(pred_data)\n",
@@ -1090,21 +465,7 @@
         "id": "lrTfXZzLsKd6",
         "outputId": "0f2c6e24-bc62-45a6-bc71-9b6918ab961e"
       },
-      "outputs": [
-        {
-          "name": "stdout",
-          "output_type": "stream",
-          "text": [
-            "[' The flight was great', ' frowning face', 'confetti ball it was bad experience']\n",
-            " The flight was great : \n",
-            " Positive\n",
-            " frowning face : \n",
-            " Negative\n",
-            "confetti ball it was bad experience : \n",
-            " Negative\n"
-          ]
-        }
-      ],
+      "outputs": [],
       "source": [
         "pred_data = pred_data[0].values.tolist()\n",
         "print(pred_data)\n",
@@ -1142,7 +503,6 @@
       "pygments_lexer": "ipython3",
       "version": "3.10.9 (tags/v3.10.9:1dd9be6, Dec  6 2022, 20:01:21) [MSC v.1934 64 bit (AMD64)]"
     },
-    "orig_nbformat": 4,
     "vscode": {
       "interpreter": {
         "hash": "497fb9213e55408ad8b1ca9a37e341ac93888d86a532670599a03e0c8054f45a"

weights.h5

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:f65c531ac62337f7a5622b09e4bdecfeb6de6975d53cbf063be98f2667458325
+oid sha256:25a8035cac413601b82f8fdae3da3d851827d2b8bf5441ef5f321afd89c2ec4c
 size 438223128