{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Deteksi Sarkasme\n",
    "Aktivitas online telah tumbuh menjadi kekuatan besar, salah satunya adalah media sosial Twitter. Twitter telah menjadi mekanisme utama dalam menerima tanggapan dari seluruh dunia. Pada sebagian besar orang Indonesia aktif dan juga ekspresif melalui tweet mereka sehingga opini mereka dapat mengandung sindiran pahit, celaan, dan bahkan dapat berupa kata-kata\n",
    "positif untuk mengungkapkan opini negatif. Dengan begitu maka dibuatlah sebuah deteksi sarkasme agar dapat membantu mengenali apakah tweet tersebut merupakan sarkasme atau bukan. \n",
    "\n",
    "Deteksi sarkasme termasuk ke dalam Analisis sentimen yang termasuk bagian dari Natural Language Processing (NLP) berkaitan dengan menemukan maksud dari opini dalam sebuah teks. Terdapat beberapa tahapan dalam membangun sebuah sistem untuk deteksi sarkasme.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dataset\n",
    "Dataset dikumpulkan secara mandiri dengan didapat dari Twitter dengan bantuan API twitter yaitu Tweepy. Data yang diperoleh dikumpulkan dan berjumlah 600 data. Untuk pelabelan setiap tweet dilakukan dengan bantuan Guru Bahasa Indonesia yang ahli dalam bidangnya."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Tweet</th>\n",
       "      <th>Label</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>@mopuci @collegemenfess Mjb, indogsat 100k 50g...</td>\n",
       "      <td>Sarkasme</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ini jaringan pas wiken knp kenceng pakee bgttt...</td>\n",
       "      <td>Sarkasme</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>@MNCPlayID sudah lebih dari 16 jam layanan int...</td>\n",
       "      <td>BukanSarkas</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>⁦@IndiHome⁩ lapor di daerah sukatani tapos dep...</td>\n",
       "      <td>BukanSarkas</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>indosat ngntd kmpa si, ada jaringan tpi browse...</td>\n",
       "      <td>BukanSarkas</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>595</th>\n",
       "      <td>Kirain jam segini jaringan baguss</td>\n",
       "      <td>BukanSarkas</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>596</th>\n",
       "      <td>@gengysi w kirain jaringan lemot, ternyata....🤣🤣😝</td>\n",
       "      <td>BukanSarkas</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>597</th>\n",
       "      <td>gedek bgt gue, jaringan tiba² ilang pas lagi u...</td>\n",
       "      <td>BukanSarkas</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>598</th>\n",
       "      <td>@FirstMediaCares Tidak ada jaringan min dari j...</td>\n",
       "      <td>BukanSarkas</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>599</th>\n",
       "      <td>Akhirnya bisa merasakan jaringan internet lagi 😭</td>\n",
       "      <td>BukanSarkas</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>600 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                 Tweet        Label\n",
       "0    @mopuci @collegemenfess Mjb, indogsat 100k 50g...     Sarkasme\n",
       "1    ini jaringan pas wiken knp kenceng pakee bgttt...     Sarkasme\n",
       "2    @MNCPlayID sudah lebih dari 16 jam layanan int...  BukanSarkas\n",
       "3    ⁦@IndiHome⁩ lapor di daerah sukatani tapos dep...  BukanSarkas\n",
       "4    indosat ngntd kmpa si, ada jaringan tpi browse...  BukanSarkas\n",
       "..                                                 ...          ...\n",
       "595                  Kirain jam segini jaringan baguss  BukanSarkas\n",
       "596  @gengysi w kirain jaringan lemot, ternyata....🤣🤣😝  BukanSarkas\n",
       "597  gedek bgt gue, jaringan tiba² ilang pas lagi u...  BukanSarkas\n",
       "598  @FirstMediaCares Tidak ada jaringan min dari j...  BukanSarkas\n",
       "599   Akhirnya bisa merasakan jaringan internet lagi 😭  BukanSarkas\n",
       "\n",
       "[600 rows x 2 columns]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "namafile=\"#tweet_label.xlsx\"\n",
    "DataTweet=pd.read_excel(namafile)\n",
    "DataTweet"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Jumlah data antara tweet berlabel Sarkasme dan BukanSarkas dapat dilihat dibawah ini.***"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfQAAAFNCAYAAAD2E503AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAkYElEQVR4nO3debhcdZ3n8fcHAgEhgEiSYTVII2OgZTECij3QTQuo0Pi0y8SlxZbFQRnculvUFlEbG7VHHXVQXKFFQHBpEaUFUaRxAYMiyKKgoAnQEFA22ZrwnT/O70IR7k1uQure5OT9ep56qup3lvpW3Tr1qd/vnFsnVYUkSVq1rTHZBUiSpMfPQJckqQcMdEmSesBAlySpBwx0SZJ6wECXJKkHDHRpFEmOSXLyZNexvJKcmOSfJruOxSV5dZILJ7uOYUhSSf5ksuvQ6stAX40luT7JvUnuSnJ7kh8m+V9JxvW+SDKrfYhNGXKdS32cJBsl+VyS/2zP51dJ3jrMulZHSfZK8lCSu9vlhiTvnuy6lmSg1rtb7fcO3H/FGMvslWTBCqzh/CT3tffmnUkuSXJUkqnLsI5J+8KQ5A1JrkvyxyRXJXnqwLSXJ/ltm/ZvSTYemDa1bZd3tm3zzZNR/+rCQNcBVTUNeDJwHPBW4LOTW9Jy+TCwPvA0YEPgr4BfL8+Khv0FpQdurKr1q2p94DnAwUleOMk1jWmk1lbv7+je8yNtX5zAUo5o29qmwFuAucC3kmQCa1hmSQ4BDgZeQLeN7Q/c2qZtD5wA/A0wE7gHOH5g8WOAbek+X/4c+Ick+01U7asbA10AVNUdVXUm8D+Bg5LsAJDkBUl+1r5hz09yzMBiF7Tr21tv51lJtkny3SS3Jbk1yReTbDSyQJK3tl7dXUl+mWTv1r5G67H8ui17+sA3/cc8zihP4ZnAKVX1h6p6qKqurqovDzzu/231j/SO/mxg2jFJvpzk5CR3Aq8eXHGStZKcmuQrSdZO8retl3JXkt8kee3AvJskOauNePw+yX+MjHi0EZG/T3JZ6818NsnMJGe3dX0nyRMH1rV7GzW5PcnPk+w11t8vyc5JftrW8yVgnYFpT2w1LUzyh3Z7i4Hp5yd5b5IftOXPSbLJWI81qKquA34IzG7resxoSlv/IWPU/cEkFybZcEnvtYH1HpTkd+299Y7x1DiW1nv8SJIb2+UjrW094GxgszzSk98sya5JftT+Hjcl+XiStZf1cavqj1V1Pt2XzmfRBSVLWn+SkW3g562e/7m0v+uK0N677wLeVFVXVufXVfX7NssrgG9U1QVVdTfwTuCvk0xr018FvLdtl1cBn2ax7UsrjoGuR6mqi4EFwEjg/ZFuo9yI7oPn8DzSG/sf7Xqj1tv5ERDgn4HN6HrLW9J9SyfJdsARwDNbT2Vf4Pq2jiOBFwJ7tmX/APy/JTzO4n4MHNvCdttRpv8E2AnYGDgFOCPJOgPTDwS+3J7nw722JOsC/wbcD7y0qh4AbqHrpWwA/C3w4SS7tEXeQvf6TafrsbwdGPx95RcBzwWeChxAFxxvBzah2x6PbI+7OfBN4J9azX8HfCXJ9MWfWPvQ/zfgC23eM9rjjFgD+DxdL2kr4F7g44ut5uXtucwA1m6Pt1Tttd6D7vUft3Rf4D4NPB3Yp6ruYMnvtRHPAbYD9gaOTvK0ZXncxbwD2J3ufbEjsCvwj1X1R+B5DIxEVNWNwCLgTXR/q2e1Gl63vA9eVb8D5vHItjbm+qtqZBvYsdXzJcb3d33YwBfN0S5njbHYFu2yQ/uSdV2Sd+eR3XLbAz8feE6/Bh4Antq+nG42OL3d3n7pr46Wh4Gu0dxIFwxU1flVdXnr9V4GnEoXuqOqqmur6tyqur+qFgIfGph/ETAVmJ1kraq6vn0AALwWeEdVLaiq++m+BLw44x/+/t90QXwEcGWSa5M8b6Cuk6vqtqp6sKr+T6tju4Hlf1RV/9ae572tbQPg3+mG7v+2qha1dX2z9VKqqr4PnMMjH8r/RTek+uSq+q+q+o969AkTPlZVN1fVDcB/ABdV1c/ac/4asHOb75XAt6rqW62mc+k+/J8/ynPfHVgL+Eh7zC/TfYEZee63VdVXquqeqroLOJbH/g0/X1W/as/9dLqQG8tmLQTuBH4FXAQsy4Fua9G9jzamG/6+p9U5nvfau6vq3qr6OV047LgMj7u4VwDvqapb2nv13XRDx6Oqqkuq6sftPXQ93VDzmNvCOA1ua8u0/nH+XQfn37+qNhrjsv8Yi430+PcB/pRu2PxldEPw0A3B37HYMncA09o0Fps+Mk1DYKBrNJsDvwdIsluS77VhvTuA/0XXgxhVkhlJTks3rH4ncPLI/FV1LfBGurC+pc23WVv0ycDXRnoMwFV0XwBmjqfg9iH/vqp6BvAkulA6I23YPslb0g2T39HWv+Fiz2P+KKvdna4HedxgKCd5XpIfpxtSv50uZEfW9UHgWuCcdMPxRy22zpsHbt87yv2RD8EnAy8Z7EXR9U43HaXOzYAbFvvi8NuBep+Q5IR0By7dSbcLY6Mkaw7M/58Dt+8ZqGM0N7YQ2ICuN30vcNIS5l/cn9CNiLy7jXiM1Dme99qy1Lk0mzHwOrXbm40xL0me2nq5/9lex/eNUt+yGtzWlmn94/y7Pl4jX24/UFW3D3zRGPlieTfdF99BGwB3tWksNn1kmobAQNejJHkm3YfMSI/rFOBMYMuq2hD4JN2wOjx6KHnEP7f2p7cP/FcOzE9VnVJVz6ELrALe3ybNB563WK9hndaTXaZTAlbVyIfhesDW6faXvxV4KfDEqtqIrqcweDDSaI9xTns+5yWZCd1+V+ArwL8AM9u6vjWyrqq6q6reUlVPoRtSf3PacQLLaD7whcVej/Wq6rhR5r0J2Dx51MFVWw3cfgvdaMRu7W8yMnz7uA/GakPlp9A9V+iGzQGeMDDbf1tssavohvfPbrthRizpvTYMN9K9D0ds1dpg9PfDJ4CrgW3b6/j2x1Nfki2BZ9CN1CzP+pfp75ruWI27x7icPcZj/JJuCH2sbfAKBkZJkjyFbvTrV1X1B7r35uAoyo5tGQ2BgS4AkmyQZH/gNODkqrq8TZoG/L6q7kuyK92+1hELgYeApwy0TaP7Zn572w/89wOPsV2Sv2iheB/dt/9FbfIn6faBP7nNOz3JgUt4nMXrf2eSZ6Y7aG0d4A3A7XQfSNOAB9t6piQ5msf2KkZVVR+gC5rz0h0otjbdB9ZC4ME2rL/PQB37J/mTFq53tue36LFrXqqTgQOS7JtkzSTrpPtXqtEOevpRe35HJpmS5K/p9gePmEb3Wt/eRizetRz1jCrJ+nRHa18B0IaubwBe2ep+DbDN4stV1al0gfWdJCPTl/ReG4ZTgX9s77VNgKPpXnfoRk6elGTDgfmn0f1N707y34HDl+dBW896T+DrwMV0XwjHs/6beey2Nu6/a1U9rwaO+F/s8rwxlrkH+BLd0enT2vvvUGBkn/sX6d6nf5buYML3AF9tuwAA/pXuNX5ie06HAicuqU4tPwNd30hyF12P8B10+7z/dmD664D3tHmOphvKBh7e2I8FftCGhXen2w+5C10P+JvAVwfWNZXuX+NupRs6nUH3oQ7wf+l6Z+e0x/oxsNsSHmdxRXeA0K10vaznAi+o7sjbb9MdfPYrumHV+xh9iH1UVfVeuoPOvkO3//fI9jr8gS50zhyYfds23910QXt8dUc0L5Oqmk83LP12ui8P8+m+HD1mm23D1n9Nd/TwH+j+U2Hwdf8IsC7da/NjuuMCHo+Hj/6mez03ptsfPeLQVuttdAdA/XC0lVTVSXQB8N0ks1jCe21pkrwiybL2/P6J7riEy4DLgZ+2NqrqarrA/017z21Gd6Dgy+mGjD9NF3TL4uPtud1M9zf5CrBfVT3Upi9t/ccAJ7V6XsqK/7uO5Qi69/ONdO/pU4DPAVTVFXS7Rr5Id7DoNB59oOC76I5B+S3wfeCDVTWsOld7efRuN0mStCqyhy5JUg8Y6JIk9YCBLklSDxjokiT1gIEuSVIPrNJnldpkk01q1qxZk12GJEkT5pJLLrm1qh5zXodVOtBnzZrFvHnzJrsMSZImTJLfjtbukLskST1goEuS1AMGuiRpVPfddx/bbbcdSTjiiCMA+NznPsc222zDuuuuy7777ssNN9zw8Py77bYb06ZN4wlPeAJz5szhggsumKzSV0tDDfQk1ye5PMmlSea1to2TnJvkmnb9xIH535buPNa/TLLvMGuTJC3Ze97zHhYsWPDw/Xnz5nHIIYew+eab8/73v5/zzz+fww9/5Bwyz372s/noRz/KO9/5Ti699FIOOeSQySh7tTURPfQ/r6qdqmpOu38UcF5VbQuc1+6TZDbdWZu2B/YDjl/B5/WVJI3TZZddxoc//GGOOeaYh9u+//3vU1W89rWv5cgjj2SXXXbhrLPO4rbbbgPgQx/6EAcccAB77703U6dOZY01HASeSJPxah8InNRunwS8cKD9tKq6v6quA67l0aeAlCRNgIceeohDDjmE17/+9Tzzmc98uH3GjBkAXHjhhVx99dVcc801VBXXX389AHfccQfTp09nt912Y+211+Yzn/nMZJS/2hp2oBfd6TAvSXJYa5tZVTcBtOsZrX1zHn1KywWtTZI0gT7/+c9z/fXX86pXverhfeR33HEH++yzD3vssQef/OQnedrTnsYDDzwAwDrrrAPA+uuvzznnnMNHP/pR7rvvPo4++uhJew6ro2H/H/oeVXVjkhnAuUmuXsK8GaXtMed2bV8MDgPYaqutVkyVkqSHzZ8/n4ULF7Ljjjs+3HbyySczdepULrjgAi6//HKmTJnCG9/4Ri688EKe8pSnADBlyhSe+9zn8tznPpcvf/nLfO973+PWW29lk002maynsloZaqBX1Y3t+pYkX6MbQr85yaZVdVOSTYFb2uwLgC0HFt8CuHGUdX4K+BTAnDlzPJm7JK1gL33pS9lhhx0AuOKKKzjmmGPYb7/9OPzww3nTm97EzjvvzE9+8hO+853v8OY3v5l1112Xb3/725x++uk8+9nPZv78+fzwhz9k5syZPOlJT5rkZ7P6GFqgJ1kPWKOq7mq39wHeA5wJHAQc166/3hY5EzglyYeAzYBtgYuHVZ8kaXSzZ89m9uzZAA/3rrfZZht23nlnDj74YE444QTWW289jjjiCN73vvcBsPHGG3PRRRdxyimnMHXqVJ7znOfwgQ98gGS0wVcNQ6qG08lN8hTga+3uFOCUqjo2yZOA04GtgN8BL6mq37dl3gG8BngQeGNVnb2kx5gzZ07506+SpNVJkksG/nPsYUProVfVb4AdR2m/Ddh7jGWOBY4dVk2SeuoUe4FaSb184vYM+0+CkiT1gIEuSVIPGOiSJPWAgS5JUg8Y6JIk9YCBLklSDxjokiT1gIEuSVIPGOiSJPWAgS5JUg8Y6JIk9YCBLklSDxjokiT1gIEuSVIPGOiSJPWAgS5JUg8Y6JIk9YCBLklSDxjokiT1gIEuSVIPGOiSJPWAgS5JUg8Y6JIk9YCBLklSDxjokiT1gIEuSVIPGOiSJPWAgS5JUg8Y6JIk9YCBLklSDxjokiT1gIEuSVIPGOiSJPWAgS5JUg8Y6JIk9YCBLklSDxjokiT1gIEuSVIPGOiSJPWAgS5JUg8Y6JIk9YCBLklSDxjokiT1gIEuSVIPDD3Qk6yZ5GdJzmr3N05ybpJr2vUTB+Z9W5Jrk/wyyb7Drk2SpL6YiB76G4CrBu4fBZxXVdsC57X7JJkNzAW2B/YDjk+y5gTUJ0nSKm+ogZ5kC+AFwGcGmg8ETmq3TwJeONB+WlXdX1XXAdcCuw6zPkmS+mLYPfSPAP8APDTQNrOqbgJo1zNa++bA/IH5FrQ2SZK0FEML9CT7A7dU1SXjXWSUthplvYclmZdk3sKFCx9XjZIk9cUwe+h7AH+V5HrgNOAvkpwM3JxkU4B2fUubfwGw5cDyWwA3Lr7SqvpUVc2pqjnTp08fYvmSJK06hhboVfW2qtqiqmbRHez23ap6JXAmcFCb7SDg6+32mcDcJFOTbA1sC1w8rPokSeqTKZPwmMcBpyc5GPgd8BKAqroiyenAlcCDwOuratEk1CdJ0ipnQgK9qs4Hzm+3bwP2HmO+Y4FjJ6ImSZL6xF+KkySpBwx0SZJ6wECXJKkHDHRJknrAQJckqQcMdEmSesBAlySpBwx0SZJ6wECXJKkHDHRJknrAQJckqQcMdEmSesBAlySpBwx0SZJ6wECXJKkHDHRJknrAQJckqQcMdEmSesBAlySpBwx0SZJ6wECXJKkHDHRJknrAQJckqQcMdEmSesBAlySpBwx0SZJ6wECXJKkHDHRJknrAQJckqQcMdEmSesBAlySpBwx0SZJ6wECXJKkHDHRJknrAQJckqQcMdEmSesBAlySpBwx0SZJ6wECXJKkHDHRJknrAQJckqQcMdEmSesBAlySpB4YW6EnWSXJxkp8nuSLJu1v7xknOTXJNu37iwDJvS3Jtkl8m2XdYtUmS1DfD7KHfD/xFVe0I7ATsl2R34CjgvKraFjiv3SfJbGAusD2wH3B8kjWHWJ8kSb0xtECvzt3t7lrtUsCBwEmt/STghe32gcBpVXV/VV0HXAvsOqz6JEnqk6HuQ0+yZpJLgVuAc6vqImBmVd0E0K5ntNk3B+YPLL6gtUmSpKUYaqBX1aKq2gnYAtg1yQ5LmD2jreIxMyWHJZmXZN7ChQtXUKWSJK3aJuQo96q6HTifbt/4zUk2BWjXt7TZFgBbDiy2BXDjKOv6VFXNqao506dPH2bZkiStMoZ5lPv0JBu12+sCfwlcDZwJHNRmOwj4ert9JjA3ydQkWwPbAhcPqz5JkvpkyhDXvSlwUjtSfQ3g9Ko6K8mPgNOTHAz8DngJQFVdkeR04ErgQeD1VbVoiPVJktQbQwv0qroM2HmU9tuAvcdY5ljg2GHVJElSX/lLcZIk9cC4euhJng3MGpy/qv51SDVJkqRltNRAT/IFYBvgUmBkn3YBBrokSSuJ8fTQ5wCzq+ox/xMuSZJWDuPZh/4L4L8NuxBJkrT8xuyhJ/kG3dD6NODKJBfTnXAFgKr6q+GXJ0mSxmNJQ+7/MmFVSJKkx2XMQK+q709kIZIkafktdR96kt2T/CTJ3UkeSLIoyZ0TUZwkSRqf8RwU93HgZcA1wLrAIa1NkiStJMb1wzJVdW2SNdtvq38+yQ+HXJckSVoG4wn0e5KsDVya5APATcB6wy1LkiQti/EMuf8NsCZwBPBHunOWv2iYRUmSpGWz1B56Vf223bwXePdwy5EkSctjST8sczndD8uMqqqePpSKJEnSMltSD33/CatCkiQ9Lkv6YZnfDt5PssGS5pckSZNnPKdPfS3wHrp96CND8AU8ZYh1SZKkZTCeHvffAdtX1a3DLkaSJC2f8fzb2q+Be4ZdiCRJWn7j6aG/Dfhhkot49OlTjxxaVZIkaZmMJ9BPAL4LXA48NNxyJEnS8hhPoD9YVW8eeiWSJGm5jWcf+veSHJZk0yQbj1yGXpkkSRq38fTQX96u3zbQ5r+tSZK0EhnPb7lvPRGFSJKk5TeeH5Z51WjtVfWvK74cSZK0PMYz5P7MgdvrAHsDPwUMdEmSVhLjGXL/34P3k2wIfGFoFUmSpGU2nqPcF3cPsO2KLkSSJC2/8exD/waPnJRlTWA2cPowi5IkSctmPPvQ/4VHAv1B4LdVdcPwSpIkSctqzEBPchddkGexSZXkfrqTtryjqs4bYn2SJGkcxgz0qpo21rQkawI7AF9s15IkaRItz0FxVNWiqvo58LEVXI8kSVoOyxXoI6rqhBVViCRJWn6PK9AlSdLKwUCXJKkHDHRJknrAQJckqQcMdEmSesBAlySpBwx0SZJ6wECXJKkHhhboSbZM8r0kVyW5IskbWvvGSc5Nck27fuLAMm9Lcm2SXybZd1i1SZLUN8PsoT8IvKWqngbsDrw+yWzgKOC8qtoWOK/dp02bC2wP7Acc334zXpIkLcXQAr2qbqqqn7bbdwFXAZsDBwIntdlOAl7Ybh8InFZV91fVdcC1wK7Dqk+SpD6ZkH3oSWYBOwMXATOr6iboQh+Y0WbbHJg/sNiC1iZJkpZi6IGeZH3gK8Abq+rOJc06SluNsr7DksxLMm/hwoUrqkxJklZpQw30JGvRhfkXq+qrrfnmJJu26ZsCt7T2BcCWA4tvAdy4+Dqr6lNVNaeq5kyfPn14xUuStAoZ5lHuAT4LXFVVHxqYdCZwULt9EPD1gfa5SaYm2RrYFrh4WPVJktQnU4a47j2AvwEuT3Jpa3s7cBxwepKDgd8BLwGoqiuSnA5cSXeE/OuratEQ65MkqTeGFuhVdSGj7xcH2HuMZY4Fjh1WTZIk9ZW/FCdJUg8Y6JIk9YCBLklSDxjokiT1gIEuSVIPGOiSJPWAgS5JUg8Y6JIk9YCBLklSDxjokiT1gIEuSVIPGOiSJPWAgS5JUg8Y6JIk9YCBLklSDxjokiT1gIEuSVIPGOiSJPWAgS5JUg8Y6JIk9YCBLklSDxjokiT1gIEuSVIPGOiSJPWAgS5JUg8Y6JIk9YCBLklSDxjokiT1gIEuSVIPGOiSJPWAgS5JUg8Y6JIk9YCBLklSDxjokiT1gIEuSVIPGOgaqiOPPJKZM2eShP333//h9h/84Ac8/elPZ+rUqeyyyy789Kc/BeDEE08kyWMu119//SQ9A0laNRjoGrq5c+c+6v59993Hi170Iu666y4+/OEPc/PNN/PiF7+YRYsWseeee3Lqqady6qmn8oUvfIG1116bmTNnsvnmm09S9ZK0ajDQNVQf/ehHedOb3vSotrPPPpubb76Z173udbzuda/j4IMP5rrrruP8889n6623Zu7cucydO5d11lmHBx54gNe85jWstdZak/QMJGnVYKBrwl133XUAD/e6t9hiCwB+85vfPGq+E044gTXWWIPDDjtsYguUpFWQga5JV1UAJHm47de//jXnnXce++23H7NmzZqkyiRp1WGga8JtvfXWACxYsACAG2644VHt0PXOq4rDDz984guUpFXQlMkuQP32zW9+k1/84hcAzJ8/n8985jPstttuzJgxg0984hNMmzaNz372s8yaNYu99toLgAceeIATTzyRrbbaiuc///mTWL0krTrsoWuoPvjBD3LUUUcBcNlll3HooYdyySWXcMYZZ7D++uvzhje8gRkzZnDGGWew5pprAvDVr36VhQsXcuihh7LGGr5FJWk8MrL/coWvOPkcsD9wS1Xt0No2Br4EzAKuB15aVX9o094GHAwsAo6sqm8v7THmzJlT8+bNG0r9klYhp2Tp80iT4eUrPmOTXFJVcxZvH2b350Rgv8XajgLOq6ptgfPafZLMBuYC27dljk+y5hBrG1PixcvKeZGkJRlaoFfVBcDvF2s+EDip3T4JeOFA+2lVdX9VXQdcC+w6rNokSeqbid5BObOqbgJo1zNa++bA/IH5FrQ2SZI0DivLEUejDSiOuuMhyWFJ5iWZt3DhwiGXJUnSqmGiA/3mJJsCtOtbWvsCYMuB+bYAbhxtBVX1qaqaU1Vzpk+fPtRiJUlaVUx0oJ8JHNRuHwR8faB9bpKpSbYGtgUunuDaJElaZQ3th2WSnArsBWySZAHwLuA44PQkBwO/A14CUFVXJDkduBJ4EHh9VS0aVm2SJPXN0AK9ql42xqS9x5j/WODYYdUjSVKfrSwHxUmSpMfBQJckqQcMdEmSesBAlySpBwx0SZJ6wECXJKkHDHRJknrAQJckqQcMdEmSesBAlySpBwx0SZJ6wECXJKkHDHRJknrAQJckqQcMdEmSesBAlySpBwx0SZJ6wECXJKkHDHRJknrAQJckqQcMdEmSesBAlySpBwx0SZJ6wECXJKkHDHRJknrAQJckqQcMdEmSesBAlySpBwx0SZJ6wECXJKkHDHRJknrAQJckqQcMdEmSesBAlySpBwx0SZJ6wECXJKkHDHRJknrAQJckqQcMdEmSesBAlySpBwx0SZJ6wECXJKkHDHRJknrAQJckqQdWukBPsl+SXya5NslRk12PJEmrgpUq0JOsCfw/4HnAbOBlSWZPblWSJK38VqpAB3YFrq2q31TVA8BpwIGTXJMkSSu9lS3QNwfmD9xf0NokSdISTJnsAhaTUdrqUTMkhwGHtbt3J/nl0KvS8toEuHWyi+iLjLZ1SMPhtruivGIoG+6TR2tc2QJ9AbDlwP0tgBsHZ6iqTwGfmsiitHySzKuqOZNdh6Rl47a7alrZhtx/AmybZOskawNzgTMnuSZJklZ6K1UPvaoeTHIE8G1gTeBzVXXFJJclSdJKb6UKdICq+hbwrcmuQyuEu0akVZPb7iooVbX0uSRJ0kptZduHLkmSloOBvppL8o4kVyS5LMmlSXZbhmVPTPLiYdYnrY6SLGrb48+T/DTJs8exzPVJNlkBj717kova41+V5JhlXH6vJGc93jq07Fa6feiaOEmeBewP7FJV97cPg7XHuazvHWl47q2qnQCS7Av8M7DnBD32ScBLq+rn7ee4txvvgn4uTC576Ku3TYFbq+p+gKq6tapuTHJ0kp8k+UWSTyXdT5okOT/J+5J8H3jD4IqSvLf12NdI8okk81rP/90D8xyX5Mo2GvAvre3ENv/3kvwmyZ5JPtd6BicOLLtPkh+13soZSdafgNdHWhlsAPwBHtv7TfLxJK8enDnJukn+PcmhSdZPcl7bbi5PcmCbZ1bbxj7dttNzkqzbVjEDuAmgqhZV1ZVtmV2T/DDJz9r1dq391W2b/AZwzmK1PLPN/5QkB7Se/8+SfCfJzDbPnm004NI2bdoQXsPVQ1V5WU0vwPrApcCvgOOBPVv7xgPzfAE4oN0+Hzh+YNqJwIuBDwAn8MhBlhu36zXbMk8HNgZ+OTDPRgPrOI3uVwIPBO4E/pTuy+YlwE50v1p1AbBeW+atwNGT/fp58TKsC7CobZtXA3cAz2jtewFnDcz3ceDV7fb1wCzgO8CrWtsUYIN2exPg2ratzQIeBHZq004HXtluH033BeJrwGuBdVr7BsCUdvsvga+026+m+1GwjQdrBJ7dtuGtWvsTB7b/Q4D/025/A9ij3V5/5DG8LPvF4ZHVWFXdneQZwJ8Bfw58qZ2y9q4k/wA8gS6Ir6Db6AC+tNhq3glcVFWHDbS9tP1E7xS6UYDZwJXAfcBnknyTboMf8Y2qqiSXAzdX1eUASa6g++DZoq3jB22wYG3gRyvgJZBWVoND7s8C/jXJDuNY7uvAB6rqi+1+gPcl+R/AQ3TnxpjZpl1XVZe225fQbWtU1XuSfBHYB3g58DK6kN4QOCnJtnQ/yb3WwOOeW1W/H7j/NLp/fdunqkZ+7XMLus+YTem24eta+w+AD7XH/GpVLRjH89QoHHJfzVU3pHZ+Vb0LOAJ4BV1v/cVV9afAp4F1Bhb542Kr+AnwjCQbAyTZGvg7YO+qejrwTbpv+A/SnU3vK8ALgX8fWMf97fqhgdsj96fQfSidW1U7tcvsqjr4cT51aZVQVT+i611Pp+tVD35ur7PY7D8Anjeym4xue55O18PfCbh5YJnBbW0RA8dUVdWvq+oTwN7AjkmeBLwX+F5V7QAcwJI/F26i+wK/80Dbx4CPt8+V144sX1XH0fXY1wV+nOS/j/liaIkM9NVYku3at+0RO9ENiwPc2vZTL+0o9n8HjgO+2fZ9bUC3cd/R9pE9rz3W+sCG1f1w0BvbY43Xj4E9kvxJW9cTkjx1GZaXVlkt4NYEbgN+C8xOMjXJhnSBO+joNt/x7f6GwC1V9V9J/pwxTuqx2OO9YOALwbZ0YX97W9cNrf3VS1nN7cAL6EYH9hqoZWT5gwYeb5uquryq3g/MAwz05eSQ++ptfeBjSTai++Z/Ld2Z7G4HLqfbJ/eTpa2kqs5oYX4m8HzgZ3TD9L+h6zEATAO+nmQduh73m8ZbZFUtbAf+nJpkamv+R7p9/1IfrZvk0nY7wEFVtQiYn+R04DLgGrptbXFvBD6X5AN0x7d8I8k8HtknvzR/A3w4yT10nwuvqKpFbX0nJXkz8N2lraSqbk5yAHB2ktcAxwBnJLmB7kv61iP1ti8bi+h2zZ09jho1Cn8pTpKkHnDIXZKkHjDQJUnqAQNdkqQeMNAlSeoBA12SpB4w0CWR5O5lmPeYJH83rPVLWj4GuiRJPWCgSxrVWGfHanZM8t0k1yQ5dGCZv093pr7LMnCmPUnDZ6BLGsuFwO5VtTPdGfH+YWDa0+l+2vNZwNFJNkuyD91Phe5K99O+z2gnBZE0AfzpV0ljGevsWABfr6p7gXuTfI8uxJ9Dd4aukZ8jXZ8u4C+YuJKl1ZeBLmksHwM+VFVnthNsHDMwbfHfjC663xz/56o6YUKqk/QoDrlLGsuoZ8dqDkyyTjut5l50J/H5NvCadmY9kmyeZMZEFSut7uyhSwJ4QpIFA/c/xNhnxwK4mO5c91sB762qG4EbkzwN+FE7++bdwCuBW4ZfviTPtiZJUg845C5JUg8Y6JIk9YCBLklSDxjokiT1gIEuSVIPGOiSJPWAgS5JUg8Y6JIk9cD/BwCBpiNScPhGAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 576x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "jumlahData=DataTweet[\"Tweet\"].shape[0] #Jumlah Data\n",
    "jmlSarkas=DataTweet[DataTweet.Label==\"Sarkasme\"].shape[0]\n",
    "jmlNonSarkas=DataTweet[DataTweet.Label==\"BukanSarkas\"].shape[0]\n",
    "d={'Jumlah': [jumlahData], 'Sarkasme': [jmlSarkas],'BukanSarkas':[jmlNonSarkas]}\n",
    "\n",
    "cekJumlah = pd.DataFrame(data=d, columns=['Sarkasme','BukanSarkas'])\n",
    "# creating the dataset\n",
    "data = {'Sarkasme':jmlSarkas, 'BukanSarkas':jmlNonSarkas}\n",
    "courses = list(data.keys())\n",
    "values = list(data.values())\n",
    "fig = plt.figure(figsize = (8, 5))\n",
    "\n",
    "# creating the bar plot\n",
    "plt.bar(courses, values, color ={'orange','blue'} ,width = 0.7)\n",
    "plt.xlabel(\"Label\")\n",
    "plt.ylabel(\"Jumlah\")\n",
    "plt.title(\"Dataset Sarkasme dan Bukan. Total Data = %s\" %jumlahData)\n",
    "for x, y in enumerate(values):\n",
    "    plt.text(x , y + 2, str(y),color = 'black', fontweight = 'bold')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Import Library Python\n",
    "Lakukan import semua library yang dibutuhkan."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "import re,string\n",
    "from sklearn.feature_extraction.text import TfidfVectorizer\n",
    "from sklearn.svm import SVC\n",
    "from sklearn.ensemble import AdaBoostClassifier\n",
    "from sklearn.model_selection import KFold\n",
    "from sklearn import model_selection\n",
    "from sklearn.metrics import accuracy_score\n",
    "from imblearn.over_sampling import SMOTE\n",
    "from collections import Counter\n",
    "from nltk.tokenize import word_tokenize\n",
    "from nltk.corpus import stopwords\n",
    "from Sastrawi.Stemmer.StemmerFactory import StemmerFactory\n",
    "import matplotlib.pyplot as plt\n",
    "from timeit import default_timer as timer\n",
    "import pickle"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Text Preprocessing\n",
    "\n",
    "Data teks termasuk data yang tidak terstruktur, sehingga diperlukan untuk mengubah data menjadi bentuk yang terstruktur, oleh karena itu proses tersebut adalah Text Preprocessing.\n",
    "\n",
    "**Fungsi yang saya buat untuk text-preprocessing**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "def lower(text):\n",
    "    # lowercase\n",
    "    lower = text.lower()\n",
    "    return lower\n",
    "\n",
    "def removeURLemoji(text):\n",
    "    # hapus hastag/mention\n",
    "    HastagRT = re.sub(r\"#(\\w+)|@(\\w+)|(\\brt\\b)\", \" \", text)\n",
    "    # hapus URL\n",
    "    pola_url = r'http\\S+'\n",
    "    CleanURL = re.sub(pola_url, \" \", HastagRT)\n",
    "    # hapus emoticon\n",
    "    hps_emoji = hapus_emoticon(CleanURL)\n",
    "    # hapus multiWhitespace++, ex: ahh   haa\n",
    "    text = re.sub('\\s+', ' ', hps_emoji)\n",
    "    # hasil akhir casefolding\n",
    "    hasil = text\n",
    "    return hasil\n",
    "\n",
    "def angkadua(teksAwal2):\n",
    "    final2 = []\n",
    "    huruf2 = \"\"\n",
    "    for x in range(len(teksAwal2)):\n",
    "        cek2 = [i for i in teksAwal2[x]]\n",
    "        for x in range(len(cek2)):\n",
    "            if x == 0:\n",
    "                final2.append(cek2[0])\n",
    "                huruf2 = cek2[0]\n",
    "            else:\n",
    "                if cek2[x] != huruf2:\n",
    "                    if cek2[x] == \"2\":\n",
    "                        if(len(final2)) == 2:\n",
    "                            final2.append(cek2[x-2])\n",
    "                            final2.append(cek2[x-1])\n",
    "                            huruf2 = cek2[x]\n",
    "                        elif(len(final2) > 2):\n",
    "                            jo = \"\".join(cek2[:2])\n",
    "                            if(jo == \"se\" or jo == \"di\"):\n",
    "                                final2.append(\" \")\n",
    "                                final2 = final2+cek2[2:x]\n",
    "                                huruf2 = cek2[x]\n",
    "                            else:\n",
    "                                final2.append(\" \")\n",
    "                                final2 = final2+cek2[:x]\n",
    "                                huruf2 = cek2[x]\n",
    "                        else:\n",
    "                            final2.append(cek2[x])\n",
    "                            huruf2 = cek2[x]\n",
    "                    else:\n",
    "                        final2.append(cek2[x])\n",
    "                        huruf2 = cek2[x]\n",
    "                else:\n",
    "                    final2.append(cek2[x])\n",
    "                    huruf2 = cek2[x]\n",
    "        final2.append(\" \")\n",
    "    hasil = \"\".join(final2).split()\n",
    "    return hasil\n",
    "\n",
    "\n",
    "def hapus_hurufganda(teksAwal):\n",
    "    jml = 0\n",
    "\n",
    "    final = []\n",
    "    huruf = \"\"\n",
    "    for x in range(len(teksAwal)):\n",
    "        cek = [i for i in teksAwal[x]]\n",
    "        for x in range(len(cek)):\n",
    "            if x == 0:\n",
    "                final.append(cek[0])\n",
    "                huruf = cek[0]\n",
    "                jml = 1\n",
    "            else:\n",
    "                if cek[x] != huruf:\n",
    "                    final.append(cek[x])\n",
    "                    huruf = cek[x]\n",
    "                    jml = 1\n",
    "                else:\n",
    "                    if jml < 2:\n",
    "                        final.append(cek[x])\n",
    "                        huruf = cek[x]\n",
    "                        jml += 1\n",
    "        final.append(\" \")\n",
    "    hasil = \"\".join(final).split()\n",
    "    return hasil\n",
    "\n",
    "\n",
    "def hapus_simbolAngka(text):\n",
    "    del_angkadua = angkadua(text)\n",
    "    del_hrfganda = hapus_hurufganda(del_angkadua)\n",
    "\n",
    "    # hasil=[]\n",
    "    token = del_hrfganda\n",
    "    lte = [\"2g\", \"3g\", \"4g\", \"5g\"]\n",
    "    for i in range(len(token)):\n",
    "        if(token[i] not in lte):\n",
    "            token[i] = re.sub(r\"\\d+\", \" \", token[i])\n",
    "\n",
    "    for ele in range(len(token)):\n",
    "        token[ele] = token[ele].translate(\n",
    "            str.maketrans('', '', string.punctuation))\n",
    "        token[ele] = re.sub('\\W', \"\", token[ele])\n",
    "        token[ele] = re.sub('\\s+', \"\", token[ele])\n",
    "\n",
    "    return token\n",
    "\n",
    "\n",
    "def hapus_simbolAngka2(text):\n",
    "    token = text\n",
    "    for i in range(len(token)):\n",
    "        cekG = re.match(r\"([\\b234]+g)\", token[i])\n",
    "        if (cekG) == None:\n",
    "            token[i] = re.sub(r\"\\d+\", \"\", token[i])\n",
    "    # initializing punctuations string\n",
    "    punc = '''!()-[]{};:'\"\\,<>./?@#$%^&*_~'''\n",
    "\n",
    "    # Removing punctuations in string\n",
    "    # Using loop + punctuation string\n",
    "    for ele in token:\n",
    "        if ele in punc:\n",
    "            token = token.replace(ele, \" \")\n",
    "            token = re.sub('\\s+', ' ', token)\n",
    "    return token\n",
    "\n",
    "\n",
    "def hapus_emoticon(text):\n",
    "    emoji_pattern = re.compile(\"[\"\n",
    "                               u\"\\U0001F600-\\U0001F64F\"  # emoticons\n",
    "                               u\"\\U0001F300-\\U0001F5FF\"  # symbols & pictographs\n",
    "                               u\"\\U0001F680-\\U0001F6FF\"  # transport & map symbols\n",
    "                               u\"\\U0001F1E0-\\U0001F1FF\"  # flags (iOS)\n",
    "                               u\"\\U00002500-\\U00002BEF\"  # chinese char\n",
    "                               u\"\\U00002702-\\U000027B0\"\n",
    "                               u\"\\U00002702-\\U000027B0\"\n",
    "                               u\"\\U000024C2-\\U0001F251\"\n",
    "                               u\"\\U0001f926-\\U0001f937\"\n",
    "                               u\"\\U00010000-\\U0010ffff\"\n",
    "                               u\"\\u2640-\\u2642\"\n",
    "                               u\"\\u2600-\\u2B55\"\n",
    "                               u\"\\u200d\"\n",
    "                               u\"\\u23cf\"\n",
    "                               u\"\\u23e9\"\n",
    "                               u\"\\u231a\"\n",
    "                               u\"\\ufe0f\"  # dingbats\n",
    "                               u\"\\u3030\"\n",
    "                               \"]+\", flags=re.UNICODE)\n",
    "    # hapus emoji\n",
    "    CleanEmoji = re.sub(emoji_pattern, \"\", text)\n",
    "    return CleanEmoji\n",
    "\n",
    "\n",
    "def tokenize(kalimat):\n",
    "    return word_tokenize(kalimat)\n",
    "\n",
    "\n",
    "def listokalimat(kalimat):\n",
    "    listToStr = ' '.join(kalimat)\n",
    "    return listToStr\n",
    "\n",
    "\n",
    "def delstopwordID(teks):\n",
    "    notsinglechar=[]\n",
    "    for kata in teks:\n",
    "        a = re.sub(r\"\\b[a-zA-Z]\\b\", \" \", kata)\n",
    "        if(a!=\" \"):\n",
    "            notsinglechar.append(a)\n",
    "    return [kata for kata in notsinglechar if kata not in list_stopwords]\n",
    "\n",
    "\n",
    "def daftarStopword():\n",
    "    list_stopwords = stopwords.words('indonesian')\n",
    "    # baca tambahan\n",
    "    my_file = open(\"_stopwordTambahan.txt\", \"r\")\n",
    "    tambahan = my_file.read()\n",
    "    daftar = tambahan.replace('\\n', ' ').split()\n",
    "    ####\n",
    "    list_stopwords.extend(daftar)\n",
    "    list_stopwords = set(list_stopwords)\n",
    "    return list_stopwords\n",
    "\n",
    "\n",
    "def normal_term():\n",
    "    normalisasi_word = pd.read_excel(\"_normalisasi.xlsx\")\n",
    "    normalisasi_dict = {}\n",
    "    for index, row in normalisasi_word.iterrows():\n",
    "        if row[0] not in normalisasi_dict:\n",
    "            normalisasi_dict[row[0]] = row[1]\n",
    "    return normalisasi_dict\n",
    "\n",
    "\n",
    "def normalisasi(document):\n",
    "    kalimat = document\n",
    "    for term in range(len(kalimat)):\n",
    "        if kalimat[term] in normalisasi_dict:\n",
    "            kalimat[term] = normalisasi_dict[kalimat[term]]\n",
    "    hasil = \" \".join(kalimat).split()\n",
    "    return hasil\n",
    "\n",
    "\n",
    "def stemming(kalimat):\n",
    "    term_dict = {}\n",
    "    for kata in kalimat:\n",
    "        for term in kalimat:\n",
    "            if term not in term_dict:\n",
    "                term_dict[term] = \" \"\n",
    "    temp = list(term_dict)\n",
    "    for x in range(len(temp)):\n",
    "        if temp[x] == \"jaringan\":\n",
    "            term_dict[temp[x]] = temp[x]\n",
    "        elif temp[x] == \"teh\" and temp[x+1] == \"anget\":\n",
    "            term_dict[temp[x]] = temp[x]\n",
    "        else:\n",
    "            term_dict[temp[x]] = stemmer.stem(temp[x])\n",
    "    kalimat = [term_dict[term] for term in kalimat]\n",
    "    #listToStr = ' '.join([str(i) for i in kalimat])\n",
    "    return kalimat\n",
    "\n",
    "\n",
    "list_stopwords = daftarStopword()\n",
    "term_dict = {}\n",
    "factory = StemmerFactory()\n",
    "stemmer = factory.create_stemmer()\n",
    "normalisasi_dict = normal_term()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***Text Preprocessing terdiri dari beberapa tahapan:***\n",
    "1. Casefolding\n",
    "Proses untuk mengubah semua huruf yang ada pada dokumen teks menjadi huruf kecil\n",
    "2. Cleaning\n",
    "Proses pembersihan karakter-karakter yang tidak diperlukan untuk mengurangi noise, seperti angka, URL, simbol yang tidak digunakan.\n",
    "3. Tokenisasi\n",
    "Pada dasarnya tokenisasi berarti memecah teks kalimat menjadi potongan kata, atau frasa yang disebut token.\n",
    "4. Normalisasi\n",
    "Proses pengubahan bentuk kata yang tidak baku atau ambigu, dan kata dengan huruf berulang-ulang yang terdapat pada dokumen teks menjadi kata yang baku. Misalnya “manisss” diubah jadi “manis”.\n",
    "5. Konversi kata gaul (Slang Word)\n",
    "Proses pengubahan kata gaul menjadi kata yang baku sesuai standar KBBI. Dalam sosial media twitter orang Indonesia terkadang menulis kata gaul pada pesan teks mereka. Sehingga untuk mengubah bentuk menjadi kata yang baku, dibutuhkan sebuah daftar kata gaul sesuai dengan yang sering digunakan pada pesan teks. Misalnya “gw” diubah jadi “saya”, “napa” diubah jadi “kenapa” dan lain sebagainya.\n",
    "\n",
    "### Casefolding dan Cleaning\n",
    "Pada pesan teks berbahasa Indonesia dari media sosial, orang sering kali menggunakan kata-kata tidak baku dari pada kata baku seperti menggunakan angka untuk mengganti alfabet, kata singkatan, karakter yang berulang, dan kata gaul. Oleh karena itu harus dilakukan tahapan preprocessing terlebih dahulu agar jadi lebih terstruktur. Selain itu akan dihilangkan tanda baca yang tidak perlu, URL yang ada dalam teks."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "##-------- Mulai Proses Preprocessing --------##\n",
      "\n",
      "\n",
      "...... Proses Casefolding lowercase, hapus URL...... \n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Tweet</th>\n",
       "      <th>Label</th>\n",
       "      <th>casefolding1</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>@mopuci @collegemenfess Mjb, indogsat 100k 50g...</td>\n",
       "      <td>Sarkasme</td>\n",
       "      <td>mjb, indogsat 100k 50gb. jaringan lebih ok ju...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ini jaringan pas wiken knp kenceng pakee bgttt...</td>\n",
       "      <td>Sarkasme</td>\n",
       "      <td>ini jaringan pas wiken knp kenceng pakee bgttt...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>@MNCPlayID sudah lebih dari 16 jam layanan int...</td>\n",
       "      <td>BukanSarkas</td>\n",
       "      <td>sudah lebih dari 16 jam layanan internet dan ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>⁦@IndiHome⁩ lapor di daerah sukatani tapos dep...</td>\n",
       "      <td>BukanSarkas</td>\n",
       "      <td>⁦ ⁩ lapor di daerah sukatani tapos depok serin...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>indosat ngntd kmpa si, ada jaringan tpi browse...</td>\n",
       "      <td>BukanSarkas</td>\n",
       "      <td>indosat ngntd kmpa si, ada jaringan tpi browse...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>595</th>\n",
       "      <td>Kirain jam segini jaringan baguss</td>\n",
       "      <td>BukanSarkas</td>\n",
       "      <td>kirain jam segini jaringan baguss</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>596</th>\n",
       "      <td>@gengysi w kirain jaringan lemot, ternyata....🤣🤣😝</td>\n",
       "      <td>BukanSarkas</td>\n",
       "      <td>w kirain jaringan lemot, ternyata....</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>597</th>\n",
       "      <td>gedek bgt gue, jaringan tiba² ilang pas lagi u...</td>\n",
       "      <td>BukanSarkas</td>\n",
       "      <td>gedek bgt gue, jaringan tiba² ilang pas lagi u...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>598</th>\n",
       "      <td>@FirstMediaCares Tidak ada jaringan min dari j...</td>\n",
       "      <td>BukanSarkas</td>\n",
       "      <td>tidak ada jaringan min dari jam 12 offline</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>599</th>\n",
       "      <td>Akhirnya bisa merasakan jaringan internet lagi 😭</td>\n",
       "      <td>BukanSarkas</td>\n",
       "      <td>akhirnya bisa merasakan jaringan internet lagi</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>600 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                 Tweet        Label  \\\n",
       "0    @mopuci @collegemenfess Mjb, indogsat 100k 50g...     Sarkasme   \n",
       "1    ini jaringan pas wiken knp kenceng pakee bgttt...     Sarkasme   \n",
       "2    @MNCPlayID sudah lebih dari 16 jam layanan int...  BukanSarkas   \n",
       "3    ⁦@IndiHome⁩ lapor di daerah sukatani tapos dep...  BukanSarkas   \n",
       "4    indosat ngntd kmpa si, ada jaringan tpi browse...  BukanSarkas   \n",
       "..                                                 ...          ...   \n",
       "595                  Kirain jam segini jaringan baguss  BukanSarkas   \n",
       "596  @gengysi w kirain jaringan lemot, ternyata....🤣🤣😝  BukanSarkas   \n",
       "597  gedek bgt gue, jaringan tiba² ilang pas lagi u...  BukanSarkas   \n",
       "598  @FirstMediaCares Tidak ada jaringan min dari j...  BukanSarkas   \n",
       "599   Akhirnya bisa merasakan jaringan internet lagi 😭  BukanSarkas   \n",
       "\n",
       "                                          casefolding1  \n",
       "0     mjb, indogsat 100k 50gb. jaringan lebih ok ju...  \n",
       "1    ini jaringan pas wiken knp kenceng pakee bgttt...  \n",
       "2     sudah lebih dari 16 jam layanan internet dan ...  \n",
       "3    ⁦ ⁩ lapor di daerah sukatani tapos depok serin...  \n",
       "4    indosat ngntd kmpa si, ada jaringan tpi browse...  \n",
       "..                                                 ...  \n",
       "595                  kirain jam segini jaringan baguss  \n",
       "596              w kirain jaringan lemot, ternyata....  \n",
       "597  gedek bgt gue, jaringan tiba² ilang pas lagi u...  \n",
       "598         tidak ada jaringan min dari jam 12 offline  \n",
       "599    akhirnya bisa merasakan jaringan internet lagi   \n",
       "\n",
       "[600 rows x 3 columns]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def lowRemoveURL(text):\n",
    "    #lowercase\n",
    "    lower=text.lower()\n",
    "    #hapus hastag/mention\n",
    "    HastagRT=re.sub(r\"#(\\w+)|@(\\w+)|(\\brt\\b)\",\" \", lower)\n",
    "    #hapus URL\n",
    "    pola_url = r'http\\S+'\n",
    "    CleanURL=re.sub(pola_url,\" \", HastagRT)\n",
    "    #hapus emoticon\n",
    "    hps_emoji=hapus_emoticon(CleanURL)\n",
    "    #hapus multiWhitespace++, ex: ahh   haa\n",
    "    text = re.sub('\\s+',' ',hps_emoji)\n",
    "    #hasil akhir casefolding\n",
    "    hasil=text\n",
    "    return hasil\n",
    "\n",
    "#============== Start Processing Text\n",
    "print(\"\\n##-------- Mulai Proses Preprocessing --------##\\n\")\n",
    "print('\\n...... Proses Casefolding lowercase, hapus URL...... ')\n",
    "DataTweet['casefolding1'] = DataTweet['Tweet'].apply(lowRemoveURL)\n",
    "DataTweet"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Tokenisasi\n",
    "Tokenisasi adalah tahapan dari teks pre-processing yang memiliki tujuan untuk memisahkan setiap teks dalam dataset menjadi potongan-potongan kata yang disebut token."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "...... Tokenisasi ...... \n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>casefolding1</th>\n",
       "      <th>Tokenisasi</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>mjb, indogsat 100k 50gb. jaringan lebih ok ju...</td>\n",
       "      <td>[mjb, ,, indogsat, 100k, 50gb, ., jaringan, le...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ini jaringan pas wiken knp kenceng pakee bgttt...</td>\n",
       "      <td>[ini, jaringan, pas, wiken, knp, kenceng, pake...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>sudah lebih dari 16 jam layanan internet dan ...</td>\n",
       "      <td>[sudah, lebih, dari, 16, jam, layanan, interne...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>⁦ ⁩ lapor di daerah sukatani tapos depok serin...</td>\n",
       "      <td>[⁦, ⁩, lapor, di, daerah, sukatani, tapos, dep...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>indosat ngntd kmpa si, ada jaringan tpi browse...</td>\n",
       "      <td>[indosat, ngntd, kmpa, si, ,, ada, jaringan, t...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>males mau bobo lagi jaringan ngajak kelahi</td>\n",
       "      <td>[males, mau, bobo, lagi, jaringan, ngajak, kel...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>hawa dingin habis hujan ditambah jaringan lem...</td>\n",
       "      <td>[hawa, dingin, habis, hujan, ditambah, jaringa...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>jaringan aku dsini lemot banget, padahal masi...</td>\n",
       "      <td>[jaringan, aku, dsini, lemot, banget, ,, padah...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>jaringan ama web gw kenapa si ngeselin banget,...</td>\n",
       "      <td>[jaringan, ama, web, gw, kenapa, si, ngeselin,...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>halo byu, sudah hampir 3 minggu ini sinyal da...</td>\n",
       "      <td>[halo, byu, ,, sudah, hampir, 3, minggu, ini, ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>hahah bru kekirim pdhl ngetweet td malem emang...</td>\n",
       "      <td>[hahah, bru, kekirim, pdhl, ngetweet, td, male...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>giliran ujian laptop sama jaringan tiba2 lemot...</td>\n",
       "      <td>[giliran, ujian, laptop, sama, jaringan, tiba2...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>jaringan kyk anyinggggg robohin aj towernya ck</td>\n",
       "      <td>[jaringan, kyk, anyinggggg, robohin, aj, tower...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                         casefolding1  \\\n",
       "0    mjb, indogsat 100k 50gb. jaringan lebih ok ju...   \n",
       "1   ini jaringan pas wiken knp kenceng pakee bgttt...   \n",
       "2    sudah lebih dari 16 jam layanan internet dan ...   \n",
       "3   ⁦ ⁩ lapor di daerah sukatani tapos depok serin...   \n",
       "4   indosat ngntd kmpa si, ada jaringan tpi browse...   \n",
       "5          males mau bobo lagi jaringan ngajak kelahi   \n",
       "6    hawa dingin habis hujan ditambah jaringan lem...   \n",
       "7    jaringan aku dsini lemot banget, padahal masi...   \n",
       "8   jaringan ama web gw kenapa si ngeselin banget,...   \n",
       "9    halo byu, sudah hampir 3 minggu ini sinyal da...   \n",
       "10  hahah bru kekirim pdhl ngetweet td malem emang...   \n",
       "11  giliran ujian laptop sama jaringan tiba2 lemot...   \n",
       "12    jaringan kyk anyinggggg robohin aj towernya ck    \n",
       "\n",
       "                                           Tokenisasi  \n",
       "0   [mjb, ,, indogsat, 100k, 50gb, ., jaringan, le...  \n",
       "1   [ini, jaringan, pas, wiken, knp, kenceng, pake...  \n",
       "2   [sudah, lebih, dari, 16, jam, layanan, interne...  \n",
       "3   [⁦, ⁩, lapor, di, daerah, sukatani, tapos, dep...  \n",
       "4   [indosat, ngntd, kmpa, si, ,, ada, jaringan, t...  \n",
       "5   [males, mau, bobo, lagi, jaringan, ngajak, kel...  \n",
       "6   [hawa, dingin, habis, hujan, ditambah, jaringa...  \n",
       "7   [jaringan, aku, dsini, lemot, banget, ,, padah...  \n",
       "8   [jaringan, ama, web, gw, kenapa, si, ngeselin,...  \n",
       "9   [halo, byu, ,, sudah, hampir, 3, minggu, ini, ...  \n",
       "10  [hahah, bru, kekirim, pdhl, ngetweet, td, male...  \n",
       "11  [giliran, ujian, laptop, sama, jaringan, tiba2...  \n",
       "12  [jaringan, kyk, anyinggggg, robohin, aj, tower...  "
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#==== Tokenisasi : memisahkan kata dalam kalimat\n",
    "print('\\n...... Tokenisasi ...... ')\n",
    "DataTweet['Tokenisasi'] = DataTweet['casefolding1'].apply(tokenize)\n",
    "DataTweet[['casefolding1','Tokenisasi']].head(13)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "...... Proses Casefolding2 hapus angka dan simbol...... \n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>casefolding2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>[mjb, , indogsat, k, gb, , jaringan, lebih, ok...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>[ini, jaringan, pas, wiken, knp, kenceng, pake...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>[sudah, lebih, dari, , jam, layanan, internet,...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>[, , lapor, di, daerah, sukatani, tapos, depok...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>[indosat, ngntd, kmpa, si, , ada, jaringan, tp...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>[males, mau, bobo, lagi, jaringan, ngajak, kel...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>[hawa, dingin, habis, hujan, ditambah, jaringa...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>[jaringan, aku, dsini, lemot, banget, , padaha...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>[jaringan, ama, web, gw, kenapa, si, ngeselin,...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>[halo, byu, , sudah, hampir, , minggu, ini, si...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>[hahah, bru, kekirim, pdhl, ngetweet, td, male...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>[giliran, ujian, laptop, sama, jaringan, tiba,...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                         casefolding2\n",
       "0   [mjb, , indogsat, k, gb, , jaringan, lebih, ok...\n",
       "1   [ini, jaringan, pas, wiken, knp, kenceng, pake...\n",
       "2   [sudah, lebih, dari, , jam, layanan, internet,...\n",
       "3   [, , lapor, di, daerah, sukatani, tapos, depok...\n",
       "4   [indosat, ngntd, kmpa, si, , ada, jaringan, tp...\n",
       "5   [males, mau, bobo, lagi, jaringan, ngajak, kel...\n",
       "6   [hawa, dingin, habis, hujan, ditambah, jaringa...\n",
       "7   [jaringan, aku, dsini, lemot, banget, , padaha...\n",
       "8   [jaringan, ama, web, gw, kenapa, si, ngeselin,...\n",
       "9   [halo, byu, , sudah, hampir, , minggu, ini, si...\n",
       "10  [hahah, bru, kekirim, pdhl, ngetweet, td, male...\n",
       "11  [giliran, ujian, laptop, sama, jaringan, tiba,..."
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print('\\n...... Proses Casefolding2 hapus angka dan simbol...... ')\n",
    "DataTweet['casefolding2'] = DataTweet['Tokenisasi'].apply(hapus_simbolAngka)\n",
    "DataTweet[['casefolding2']].head(12)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Normalisasi\n",
    "Dalam media sosial twitter orang Indonesia terkadang menulis kata gaul dan kata yang tidak baku, bahkan sering kali ada kata-kata yang salah dalam penulisannya, sehingga diperlukan untuk mengubahnya menjadi kata yang baku sesuai dengan KBBI pada tahapan text pre-processing, hal tersebut dapat disebut dengan normalisasi."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "...... Proses Normalisasi ...... \n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>casefolding2</th>\n",
       "      <th>Normalisasi</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>[maaf gabung bareng, , indosat, ribu, gigabyte...</td>\n",
       "      <td>[maaf, gabung, bareng, indosat, ribu, gigabyte...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>[ini, jaringan, waktu, minggu, kenapa, cepat, ...</td>\n",
       "      <td>[ini, jaringan, waktu, minggu, kenapa, cepat, ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>[sudah, lebih, dari, , jam, layanan, internet,...</td>\n",
       "      <td>[sudah, lebih, dari, jam, layanan, internet, d...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>[, , lapor, di, daerah, sukatani, tapos, depok...</td>\n",
       "      <td>[lapor, di, daerah, sukatani, tapos, depok, se...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>[indosat, ngentot, kenapa, sih, , ada, jaringa...</td>\n",
       "      <td>[indosat, ngentot, kenapa, sih, ada, jaringan,...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>[malas, mau, tidur, lagi, jaringan, ngajak, ke...</td>\n",
       "      <td>[malas, mau, tidur, lagi, jaringan, ngajak, ke...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>[hawa, dingin, habis, hujan, ditambah, jaringa...</td>\n",
       "      <td>[hawa, dingin, habis, hujan, ditambah, jaringa...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>[jaringan, aku, disini, lambat, banget, , pada...</td>\n",
       "      <td>[jaringan, aku, disini, lambat, banget, padaha...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>[jaringan, sama, web, aku, kenapa, sih, ngesel...</td>\n",
       "      <td>[jaringan, sama, web, aku, kenapa, sih, ngesel...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>[halo, telkomsel, , sudah, hampir, , minggu, i...</td>\n",
       "      <td>[halo, telkomsel, sudah, hampir, minggu, ini, ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>[haha, baru, kirim, padahal, tweet, tadi, mala...</td>\n",
       "      <td>[haha, baru, kirim, padahal, tweet, tadi, mala...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>[giliran, ujian, laptop, sama, jaringan, tiba,...</td>\n",
       "      <td>[giliran, ujian, laptop, sama, jaringan, tiba,...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                         casefolding2  \\\n",
       "0   [maaf gabung bareng, , indosat, ribu, gigabyte...   \n",
       "1   [ini, jaringan, waktu, minggu, kenapa, cepat, ...   \n",
       "2   [sudah, lebih, dari, , jam, layanan, internet,...   \n",
       "3   [, , lapor, di, daerah, sukatani, tapos, depok...   \n",
       "4   [indosat, ngentot, kenapa, sih, , ada, jaringa...   \n",
       "5   [malas, mau, tidur, lagi, jaringan, ngajak, ke...   \n",
       "6   [hawa, dingin, habis, hujan, ditambah, jaringa...   \n",
       "7   [jaringan, aku, disini, lambat, banget, , pada...   \n",
       "8   [jaringan, sama, web, aku, kenapa, sih, ngesel...   \n",
       "9   [halo, telkomsel, , sudah, hampir, , minggu, i...   \n",
       "10  [haha, baru, kirim, padahal, tweet, tadi, mala...   \n",
       "11  [giliran, ujian, laptop, sama, jaringan, tiba,...   \n",
       "\n",
       "                                          Normalisasi  \n",
       "0   [maaf, gabung, bareng, indosat, ribu, gigabyte...  \n",
       "1   [ini, jaringan, waktu, minggu, kenapa, cepat, ...  \n",
       "2   [sudah, lebih, dari, jam, layanan, internet, d...  \n",
       "3   [lapor, di, daerah, sukatani, tapos, depok, se...  \n",
       "4   [indosat, ngentot, kenapa, sih, ada, jaringan,...  \n",
       "5   [malas, mau, tidur, lagi, jaringan, ngajak, ke...  \n",
       "6   [hawa, dingin, habis, hujan, ditambah, jaringa...  \n",
       "7   [jaringan, aku, disini, lambat, banget, padaha...  \n",
       "8   [jaringan, sama, web, aku, kenapa, sih, ngesel...  \n",
       "9   [halo, telkomsel, sudah, hampir, minggu, ini, ...  \n",
       "10  [haha, baru, kirim, padahal, tweet, tadi, mala...  \n",
       "11  [giliran, ujian, laptop, sama, jaringan, tiba,...  "
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#============== Normalisasi: kata gaul, singkatan jadi kata baku\n",
    "print('\\n...... Proses Normalisasi ...... ')\n",
    "DataTweet['Normalisasi'] = DataTweet['casefolding2'].apply(normalisasi)\n",
    "DataTweet[['casefolding2','Normalisasi']].head(12)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Stopword Removal\n",
    "Pada tahapan ini bertujuan untuk membersihkan teks dari kata yang sangat sering digunakan sehingga muncul dalam jumah besar dan kata yang dianggap tidak memiliki makna, seperti kata ganti, penghubung dan karakter satu huruf yang tidak memiliki arti."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "...... Proses Stopword Removal ...... \n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Stopword</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>[maaf, gabung, bareng, indosat, ribu, gigabyte...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>[jaringan, minggu, cepat, pakai, banget, oi, g...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>[jam, layanan, internet, televisi, jaringan]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>[lapor, daerah, sukatani, tapos, depok, indiho...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>[indosat, ngentot, sih, jaringan, browser, jalan]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>[malas, tidur, jaringan, ngajak, kelahi]</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                            Stopword\n",
       "0  [maaf, gabung, bareng, indosat, ribu, gigabyte...\n",
       "1  [jaringan, minggu, cepat, pakai, banget, oi, g...\n",
       "2       [jam, layanan, internet, televisi, jaringan]\n",
       "3  [lapor, daerah, sukatani, tapos, depok, indiho...\n",
       "4  [indosat, ngentot, sih, jaringan, browser, jalan]\n",
       "5           [malas, tidur, jaringan, ngajak, kelahi]"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#==== Stopword Removal : hapus kata yang tidak terlalu penting\n",
    "print('\\n...... Proses Stopword Removal ...... ')\n",
    "DataTweet['Stopword'] = DataTweet['Normalisasi'].apply(delstopwordID)\n",
    "DataTweet[['Stopword']].head(6)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Stemming\n",
    "Stemming adalah tahapan dari teks pre-processing yang memiliki tujuan untuk mereduksi kata ke dalam bentuk akar atau bentuk kata dasarnya. Stemming akan mengubah kata menjadi kata dasarnya dengan menghilangkan semua imbuhan yang terdiri dari awalan, sisipan dan akhiran."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "................ Proses Stemming ................ \n",
      "0    [maaf, gabung, bareng, indosat, ribu, gigabyte...\n",
      "1    [jaringan, minggu, cepat, pakai, banget, oi, g...\n",
      "2           [jam, layan, internet, televisi, jaringan]\n",
      "Name: Stemmed, dtype: object\n",
      "\n",
      "==========\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0     maaf gabung bareng indosat ribu gigabyte jarin...\n",
       "1     jaringan minggu cepat pakai banget oi gilir ku...\n",
       "2                  jam layan internet televisi jaringan\n",
       "3     lapor daerah sukatani tapos depok indihome min...\n",
       "4            indosat ngentot sih jaringan browser jalan\n",
       "5                    malas tidur jaringan ngajak kelahi\n",
       "6     hawa dingin habis hujan tambah jaringan lambat...\n",
       "7                           jaringan lambat banget pagi\n",
       "8            jaringan web sih ngeselin banget coba coba\n",
       "9     halo telkomsel minggu sinyal jaringan internet...\n",
       "10         haha kirim tweet malam emang anjing jaringan\n",
       "11              gilir uji laptop jaringan lambat anjing\n",
       "Name: newTweet, dtype: object"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#==== Stemming : mengurangi dimensi fitur kata/term\n",
    "print('\\n................ Proses Stemming ................ ')\n",
    "DataTweet['Stemmed'] = DataTweet['Stopword'].apply(stemming)\n",
    "print(DataTweet['Stemmed'].head(3))\n",
    "DataTweet['newTweet'] = DataTweet['Stemmed'].apply(listokalimat)\n",
    "print('\\n==========')\n",
    "DataTweet['newTweet'].head(12)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Vektorisasi (TF-IDF)\n",
    "\n",
    "Setelah selesai dalam tahap teks preprocessing, data yang dihasilkan masih berupa token potongan-potongan kata yang tidak dapat langsung diproses oleh mesin untuk dilanjutkan tahap klasifikasi. Oleh karena itu diperlukan proses vektorisasi yang berguna untuk mengubah kata menjadi angka dan disusun dalam bentuk matriks dengan cara menghitung berapa kali kemunculan kata pada setiap tweet dalam dokumen yang biasa disebut pembobotan kata. Proses pembobotan kata yang paling umum digunakan adalah menggunakan TF-IDF. Hasil penerapan pembobotan kata dengan TF-IDF ditampilkan sebagai berikut:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "................ Hitung TF-IDF ................ \n",
      "         term  rata2bobot\n",
      "331  jaringan    0.113028\n",
      "340     jelek    0.051901\n",
      "73     banget    0.048090\n",
      "426    lambat    0.035820\n",
      "861        ya    0.030943\n",
      "313  internet    0.028287\n",
      "306  indihome    0.027122\n",
      "\n",
      "================\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>aba</th>\n",
       "      <th>abad</th>\n",
       "      <th>adik</th>\n",
       "      <th>admin</th>\n",
       "      <th>administrator</th>\n",
       "      <th>aduh</th>\n",
       "      <th>ah</th>\n",
       "      <th>ahli</th>\n",
       "      <th>aja</th>\n",
       "      <th>ajak</th>\n",
       "      <th>...</th>\n",
       "      <th>wkwkwk</th>\n",
       "      <th>wow</th>\n",
       "      <th>xd</th>\n",
       "      <th>xl</th>\n",
       "      <th>ya</th>\n",
       "      <th>yakxd</th>\n",
       "      <th>yaudah</th>\n",
       "      <th>youtube</th>\n",
       "      <th>yuk</th>\n",
       "      <th>zoom</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.235527</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>595</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>596</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>597</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>598</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.473133</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>599</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>600 rows × 867 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     aba  abad  adik     admin  administrator  aduh   ah  ahli  aja  ajak  \\\n",
       "0    0.0   0.0   0.0  0.000000            0.0   0.0  0.0   0.0  0.0   0.0   \n",
       "1    0.0   0.0   0.0  0.000000            0.0   0.0  0.0   0.0  0.0   0.0   \n",
       "2    0.0   0.0   0.0  0.000000            0.0   0.0  0.0   0.0  0.0   0.0   \n",
       "3    0.0   0.0   0.0  0.000000            0.0   0.0  0.0   0.0  0.0   0.0   \n",
       "4    0.0   0.0   0.0  0.000000            0.0   0.0  0.0   0.0  0.0   0.0   \n",
       "..   ...   ...   ...       ...            ...   ...  ...   ...  ...   ...   \n",
       "595  0.0   0.0   0.0  0.000000            0.0   0.0  0.0   0.0  0.0   0.0   \n",
       "596  0.0   0.0   0.0  0.000000            0.0   0.0  0.0   0.0  0.0   0.0   \n",
       "597  0.0   0.0   0.0  0.000000            0.0   0.0  0.0   0.0  0.0   0.0   \n",
       "598  0.0   0.0   0.0  0.473133            0.0   0.0  0.0   0.0  0.0   0.0   \n",
       "599  0.0   0.0   0.0  0.000000            0.0   0.0  0.0   0.0  0.0   0.0   \n",
       "\n",
       "     ...    wkwkwk  wow   xd   xl   ya  yakxd  yaudah  youtube  yuk  zoom  \n",
       "0    ...  0.235527  0.0  0.0  0.0  0.0    0.0     0.0      0.0  0.0   0.0  \n",
       "1    ...  0.000000  0.0  0.0  0.0  0.0    0.0     0.0      0.0  0.0   0.0  \n",
       "2    ...  0.000000  0.0  0.0  0.0  0.0    0.0     0.0      0.0  0.0   0.0  \n",
       "3    ...  0.000000  0.0  0.0  0.0  0.0    0.0     0.0      0.0  0.0   0.0  \n",
       "4    ...  0.000000  0.0  0.0  0.0  0.0    0.0     0.0      0.0  0.0   0.0  \n",
       "..   ...       ...  ...  ...  ...  ...    ...     ...      ...  ...   ...  \n",
       "595  ...  0.000000  0.0  0.0  0.0  0.0    0.0     0.0      0.0  0.0   0.0  \n",
       "596  ...  0.000000  0.0  0.0  0.0  0.0    0.0     0.0      0.0  0.0   0.0  \n",
       "597  ...  0.000000  0.0  0.0  0.0  0.0    0.0     0.0      0.0  0.0   0.0  \n",
       "598  ...  0.000000  0.0  0.0  0.0  0.0    0.0     0.0      0.0  0.0   0.0  \n",
       "599  ...  0.000000  0.0  0.0  0.0  0.0    0.0     0.0      0.0  0.0   0.0  \n",
       "\n",
       "[600 rows x 867 columns]"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#====================== lakukan TF-IDF\n",
    "print('\\n................ Hitung TF-IDF ................ ')\n",
    "tfidf_vect = TfidfVectorizer()\n",
    "vect_docs = tfidf_vect.fit_transform(DataTweet['newTweet'])\n",
    "#print(vect_docs)\n",
    "features_names = tfidf_vect.get_feature_names_out()\n",
    "\n",
    "datane = []\n",
    "means=vect_docs.mean(axis=0)\n",
    "for col, term in enumerate(features_names):\n",
    "    datane.append( (term, means[0,col] ))\n",
    "\n",
    "ranking = pd.DataFrame(datane, columns=['term','rata2bobot'])\n",
    "ranking = ranking.sort_values('rata2bobot', ascending=False)\n",
    "print(ranking.head(7))\n",
    "\n",
    "dense = vect_docs.todense()\n",
    "alist = dense.tolist()\n",
    "print('\\n================')\n",
    "newData = pd.DataFrame(alist,columns=features_names)\n",
    "newData"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## SMOTE\n",
    "Ketidakseimbangan kelas (imbalance class) dapat mengganggu kemampuan prediksi algoritma klasifikasi karena algoritma mengejar\n",
    "akurasi klasifikasi secara keseluruhan. Data dikatakan seimbang apabila perbandingan antara kedua kelas adalah 1:1. Masalah kelas data yang tidak simbang dapat berakibat pada ketepatan klasifikasi kelas minoritas. \n",
    "\n",
    "Untuk memecahkan masalah klasifikasi yang sulit ketika berhadapan dengan kumpulan data yang tidak seimbang, dapat menggunakan metode menyeimbangkan jumlah data pada kelas yang berbeda dengan menambahkan sampel ke kelas minoritas (oversampling) atau menghapus sampel dari kelas mayoritas (undersampling). Berikut adalah penerapan SMOTE."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "fold-berjumlah: 10\n",
      "Jumlah Data latih sebelum SMOTE = 540\n",
      "Sarkasme = 93 BukanSarkas = 447\n",
      "Jumlah Data latih setelah SMOTE = 894\n",
      "Sarkasme = 447 BukanSarkas = 447\n",
      "Jumlah Data Uji = 60\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfQAAAFNCAYAAAD2E503AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAZnUlEQVR4nO3dfbRddX3n8ffHoICGB1MCpQkatIwarGJFxIcKiqMwPoQ1IxYfw/gAdrTVUWuhTinqQpHl2HGpqLR1CGjFuNABtD7QCHVUBIKAEShDBIQIhUTlSWiU8J0/9r71eL03ObnJubn3l/drrbPO3r/923t/z1nZ93P2Q/ZOVSFJkma3h2zrAiRJ0pYz0CVJaoCBLklSAwx0SZIaYKBLktQAA12SpAYY6JI2S5JK8vtD9FvU991hOuqStncGurSFktyU5P4k9yS5M8l3k7wpyVDb19YIviRLklyZ5O4k65KsSLJoOta9tSRZmOScvv67kqxKckw/bazO74+bZ48kv0xy07j2Y/r570vyr0k+kWT3ftonk9zbv36Z5FcD418dWNe9415/PE1fhTQlBrq0dbykqnYBHg2cAvwF8PfTseJ+b/lM4B3AbsC+wGnAg9Ox/q3oLOAWuu/wd4DXAreP6/OIJE8cGH8lcONghyTvAD4I/Dnd93Fwv8wLkjysqt5UVXOrai7wfuDzY+NVdcTAonYfaJ9bVZ/fip9V2uoMdGkrqqq7quo84I+BpWPhk+RFSa7o96BvSXLSwGzf6t/v7PcEn5HksUm+meSn/R7rZ8f2MCdwAHBjVa2ozj1VdU5V3dyv+yFJjk/yo355y5PMm2zd/TyvS3Jtkp8n+XqSR0+04k18rjGvSnJz/znevZGv72nAGVX1i6p6oKquqKqvjutzFrB0YPy1dD9mxurZFXgP8KdV9bWq+lVV3QS8nC7UX72R9UuzmoEujUBVXQqsAf6ob/oFXfjsDrwI+JMkR/bTntO/j+0RXgwE+ADwe8ATgH2AkyZZ3feBxyf5myTPTTJ33PQ/A44EDumX93Pg45Otu6/rL4H/DMwH/i/wuUnWvbHPNebZwOOAw4ATkzxhkmV9D/h4kqOTPGqSPp8Bjk4yp1/OLsAlA9OfCewEfHFwpqq6F/gq8B8nWa406xno0ujcCswDqKqLqmpVVT1YVT+gC8hDJpuxqlZX1QVVtb6q1gIfnqx/Vd0AHAosAJYD65KcMRDsxwHvrqo1VbWe7ofByzZy3vw44ANVdW1VPUB3WPqAifbSh/xc76mq+6vqKuAq4MmTrPcouh8PfwXc2F8T8LRxfdYA1wHPp9tTP3Pc9D2AdX3d493WTx/Wuv6aiLHXZD9EpBnBQJdGZwHwM4AkT09yYZK1Se4C3sRGwiXJnknOTvKTJHfT7ZlO2r+qvldVL6+q+XRHBZ4DjB3efjTwpbFgAq4FNgB7TbK4RwMfGej/M7ojBgsmqHOYz/WvA8P3AeOPIIx9hp9X1fFVtX9f25XA/0mScV3PBI4BXkH3vQxaB+wxyY+Vvfvpw9qjqnYfeF27GfNK085Al0ag37NcAHy7b/oH4Dxgn6raDfgkXUgCTPTIww/07U+qql3pzv2OD7YJVdVldIecxy4euwU4Ylw47VRVP5lk3bcAx43rv3NVfXeCvhv7XFNWVeuAD9GdIpg3bvI5dIf3b6iqH4+bdjGwnu50wb9L8gjgCGDFltYmzVQGurQVJdk1yYuBs4HPVNWqftIuwM+q6t+SHER3dfaYtXRXpD9moG0X4F66i9UW0F2xPdk6n53kjUn27McfD7yU7pw0dCF78tgh8yTzkyzZyLo/CZyQZP++/25Jjppk9Rv7XJslyQeTPDHJDkl2Af4EWF1VPx3sV1W/AJ4HvGH8MqrqLrqL4j6a5PAkD0333/e+QHe4/qyp1ifNdAa6tHWcn+Qeur3bd9Od8/6vA9P/G/Devs+JdOe6Aaiq+4CTge/0h7kPpgulPwTuAr7CuIu8xrmTLsBXJbkX+BrwJeDUfvpH6Paiv9Gv/3vA0ydbd1V9ie6/fZ3dH+7/Id3e7UQm/VxT8PC+7juBG+gO/b90oo5VtbKqfjTJtFPpLur7EHA33UVztwCH9dcQDGvsyv+x19s3Y15p2qVqoiNukiRpNnEPXZKkBhjokiQ1wECXJKkBBrokSQ0w0CVJasA2f2Tilthjjz1q0aJF27oMSZKmzeWXX76uvyvkb5jVgb5o0SJWrly5rcuQJGnaJBl/h0TAQ+6SJDXBQJckqQEGuiRJDTDQJUlqgIEuSVIDDHRJkhpgoEuS1AADXZKkBhjokiQ1wECXJKkBBrokSQ2Y1fdyH4l/yLauQJrYK2tbVzBjxc1WM1RN42brHrokSQ0w0CVJaoCBLklSAwx0SZIaYKBLktQAA12SpAYY6JIkNcBAlySpAQa6JEkNMNAlSWqAgS5JUgMMdEmSGmCgS5LUAANdkqQGGOiSJDXAQJckqQEGuiRJDTDQJUlqgIEuSVIDDHRJkhpgoEuS1AADXZKkBow80JPMSXJFki/34/OSXJDk+v79kQN9T0iyOsl1SV446tokSWrFdOyhvxW4dmD8eGBFVe0HrOjHSbIYOBrYHzgcOC3JnGmoT5KkWW+kgZ5kIfAi4O8GmpcAy/rhZcCRA+1nV9X6qroRWA0cNMr6JElqxaj30P8X8C7gwYG2varqNoD+fc++fQFwy0C/NX2bJEnahJEFepIXA3dU1eXDzjJBW02w3GOTrEyycu3atVtUoyRJrRjlHvqzgJcmuQk4G3heks8AtyfZG6B/v6PvvwbYZ2D+hcCt4xdaVadX1YFVdeD8+fNHWL4kSbPHyAK9qk6oqoVVtYjuYrdvVtWrgfOApX23pcC5/fB5wNFJdkyyL7AfcOmo6pMkqSU7bIN1ngIsT/J64GbgKICqujrJcuAa4AHgzVW1YRvUJ0nSrDMtgV5VFwEX9cM/BQ6bpN/JwMnTUZMkSS3xTnGSJDXAQJckqQEGuiRJDTDQJUlqgIEuSVIDDHRJkhpgoEuS1AADXZKkBhjokiQ1wECXJKkBBrokSQ0w0CVJaoCBLklSAwx0SZIaYKBLktQAA12SpAYY6JIkNcBAlySpAQa6JEkNMNAlSWqAgS5JUgMMdEmSGmCgS5LUAANdkqQGGOiSJDXAQJckqQEGuiRJDTDQJUlqgIEuSVIDDHRJkhpgoEuS1AADXZKkBhjokiQ1wECXJKkBBrokSQ0w0CVJaoCBLklSAwx0SZIaYKBLktQAA12SpAYY6JIkNcBAlySpAQa6JEkNMNAlSWqAgS5JUgMMdEmSGmCgS5LUAANdkqQGGOiSJDVgZIGeZKcklya5KsnVSd7Tt89LckGS6/v3Rw7Mc0KS1UmuS/LCUdUmSVJrRrmHvh54XlU9GTgAODzJwcDxwIqq2g9Y0Y+TZDFwNLA/cDhwWpI5I6xPkqRmjCzQq3NvP/rQ/lXAEmBZ374MOLIfXgKcXVXrq+pGYDVw0KjqkySpJSM9h55kTpIrgTuAC6rqEmCvqroNoH/fs+++ALhlYPY1fZskSdqEkQZ6VW2oqgOAhcBBSZ64ke6ZaBG/1Sk5NsnKJCvXrl27lSqVJGl2m5ar3KvqTuAiunPjtyfZG6B/v6PvtgbYZ2C2hcCtEyzr9Ko6sKoOnD9//ijLliRp1hjlVe7zk+zeD+8MPB/4F+A8YGnfbSlwbj98HnB0kh2T7AvsB1w6qvokSWrJDiNc9t7Asv5K9YcAy6vqy0kuBpYneT1wM3AUQFVdnWQ5cA3wAPDmqtowwvokSWrGyAK9qn4APGWC9p8Ch00yz8nAyaOqSZKkVnmnOEmSGmCgS5LUAANdkqQGGOiSJDXAQJckqQEGuiRJDTDQJUlqgIEuSVIDDHRJkhpgoEuS1AADXZKkBhjokiQ1wECXJKkBBrokSQ0w0CVJaoCBLklSAwx0SZIasMMwnZI8E1g02L+qzhxRTZIkaTNtMtCTnAU8FrgS2NA3F2CgS5I0Qwyzh34gsLiqatTFSJKkqRnmHPoPgd8ddSGSJGnqJt1DT3I+3aH1XYBrklwKrB+bXlUvHX15kiRpGBs75P6haatCkiRtkUkDvar+eToLkSRJU7fJc+hJDk5yWZJ7k/wyyYYkd09HcZIkaTjDXBT3MeAVwPXAzsAb+jZJkjRDDHVjmapanWROVW0A/neS7464LkmStBmGCfT7kjwMuDLJqcBtwCNGW5YkSdocwxxyfw0wB3gL8AtgH+C/jLIoSZK0eTa5h15VP+4H7wfeM9pyJEnSVGzsxjKr6G4sM6GqetJIKpIkSZttY3voL562KiRJ0hbZ2I1lfjw4nmTXjfWXJEnbzjCPTz0OeC/dOfSxQ/AFPGaEdUmSpM0wzB73O4H9q2rdqIuRJElTM8x/W/sRcN+oC5EkSVM3zB76CcB3k1zCbz4+9c9GVpUkSdoswwT6p4BvAquAB0dbjiRJmophAv2Bqnr7yCuRJElTNsw59AuTHJtk7yTzxl4jr0ySJA1tmD30V/bvJwy0+d/WJEmaQYa5l/u+01GIJEmaumFuLPPaidqr6sytX44kSZqKYQ65P21geCfgMOD7gIEuSdIMMcwh9z8dHE+yG3DWyCqSJEmbbZir3Me7D9hvaxciSZKmbphz6Ofz64eyzAEWA8tHWZQkSdo8w5xD/xC/DvQHgB9X1U9GV5IkSdpckwZ6knvogjzjJlWS9XQPbXl3Va0YYX2SJGkIk55Dr6pdqmrX/n3wtSvwu8BxwEcmmz/JPkkuTHJtkquTvLVvn5fkgiTX9++PHJjnhCSrk1yX5IVb8XNKktS0qVwUR1VtqKqrgI9upNsDwDuq6gnAwcCbkywGjgdWVNV+wIp+nH7a0cD+wOHAaUnmTKU+SZK2N1MK9DFV9amNTLutqr7fD98DXAssAJYAy/puy4Aj++ElwNlVtb6qbgRWAwdtSX2SJG0vtijQh5VkEfAU4BJgr6q6DbrQB/bsuy0AbhmYbU3fJkmSNmHkgZ5kLnAO8LaquntjXSdoq9/q1D35bWWSlWvXrt1aZUqSNKuNNNCTPJQuzD9bVV/sm29Psnc/fW/gjr59DbDPwOwLgVvHL7OqTq+qA6vqwPnz54+ueEmSZpGRBXqSAH8PXFtVHx6YdB6wtB9eCpw70H50kh2T7Et3N7pLR1WfJEktGebGMlP1LOA1wKokV/ZtfwmcAixP8nrgZuAogKq6Osly4Bq6K+TfXFUbRlifJEnNGFmgV9W3mfi8OHRPbJtonpOBk0dVkyRJrZqWq9wlSdJoGeiSJDXAQJckqQEGuiRJDTDQJUlqgIEuSVIDDHRJkhpgoEuS1AADXZKkBhjokiQ1wECXJKkBBrokSQ0w0CVJaoCBLklSAwx0SZIaYKBLktQAA12SpAYY6JIkNcBAlySpAQa6JEkNMNAlSWqAgS5JUgMMdEmSGmCgS5LUAANdkqQGGOiSJDXAQJckqQEGuiRJDTDQJUlqgIEuSVIDDHRJkhpgoEuS1AADXZKkBhjokiQ1wECXJKkBBrokSQ0w0CVJaoCBLklSAwx0SZIaYKBLktQAA12SpAYY6JIkNcBAlySpAQa6JEkNMNAlSWqAgS5JUgMMdEmSGmCgS5LUAANdkqQGjCzQk3w6yR1JfjjQNi/JBUmu798fOTDthCSrk1yX5IWjqkuSpBaNcg/9DODwcW3HAyuqaj9gRT9OksXA0cD+/TynJZkzwtokSWrKyAK9qr4F/Gxc8xJgWT+8DDhyoP3sqlpfVTcCq4GDRlWbJEmtme5z6HtV1W0A/fueffsC4JaBfmv6NkmSNISZclFcJmirCTsmxyZZmWTl2rVrR1yWJEmzw3QH+u1J9gbo3+/o29cA+wz0WwjcOtECqur0qjqwqg6cP3/+SIuVJGm2mO5APw9Y2g8vBc4daD86yY5J9gX2Ay6d5tokSZq1dhjVgpN8DjgU2CPJGuCvgVOA5UleD9wMHAVQVVcnWQ5cAzwAvLmqNoyqNkmSWjOyQK+qV0wy6bBJ+p8MnDyqeiRJatlMuShOkiRtAQNdkqQGGOiSJDXAQJckqQEGuiRJDTDQJUlqgIEuSVIDDHRJkhpgoEuS1AADXZKkBhjokiQ1wECXJKkBBrokSQ0w0CVJaoCBLklSAwx0SZIaYKBLktQAA12SpAYY6JIkNcBAlySpAQa6JEkNMNAlSWqAgS5JUgMMdEmSGmCgS5LUAANdkqQGGOiSJDXAQJckqQEGuiRJDTDQJUlqgIEuSVIDDHRJkhpgoEuS1AADXZKkBhjokiQ1wECXJKkBBrokSQ0w0CVJaoCBLklSAwx0SZIaYKBLktQAA12SpAYY6JIkNcBAlySpAQa6JEkNMNAlSWqAgS5JUgMMdEmSGmCgS5LUgBkX6EkOT3JdktVJjt/W9UiSNBvMqEBPMgf4OHAEsBh4RZLF27YqSZJmvhkV6MBBwOqquqGqfgmcDSzZxjVJkjTjzbRAXwDcMjC+pm+TJEkbscO2LmCcTNBWv9EhORY4th+9N8l1I69KU7UHsG5bF9GMV020eUgj4ba7lWQ0m+2jJ2qcaYG+BthnYHwhcOtgh6o6HTh9OovS1CRZWVUHbus6JG0et93ZaaYdcr8M2C/JvkkeBhwNnLeNa5IkacabUXvoVfVAkrcAXwfmAJ+uqqu3cVmSJM14MyrQAarqH4F/3NZ1aKvw1Ig0O7ntzkKpqk33kiRJM9pMO4cuSZKmwEDfziV5d5Krk/wgyZVJnr4Z856R5GWjrE/aHiXZ0G+PVyX5fpJnDjHPTUn22ArrPjjJJf36r01y0mbOf2iSL29pHdp8M+4cuqZPkmcALwb+sKrW938MHjbkvP7bkUbn/qo6ACDJC4EPAIdM07qXAS+vqqv623E/btgZ/buwbbmHvn3bG1hXVesBqmpdVd2a5MQklyX5YZLTk+7WCEkuSvL+JP8MvHVwQUne1++xPyTJJ5Ks7Pf83zPQ55Qk1/RHAz7Ut53R978wyQ1JDkny6X7P4IyBeV+Q5OJ+b+ULSeZOw/cjzQS7Aj+H3977TfKxJMcMdk6yc5KvJXljkrlJVvTbzaokS/o+i/pt7G/77fQbSXbuF7EncBtAVW2oqmv6eQ5K8t0kV/Tvj+vbj+m3yfOBb4yr5Wl9/8ckeUm/539Fkn9Kslff55D+aMCV/bRdRvAdbh+qytd2+gLmAlcC/w84DTikb5830Ocs4CX98EXAaQPTzgBeBpwKfIpfX2Q5r3+f08/zJGAecN1An90HlnE23V0ClwB3A39A92PzcuAAurtWfQt4RD/PXwAnbuvvz5evUb2ADf22+S/AXcBT+/ZDgS8P9PsYcEw/fBOwCPgn4LV92w7Arv3wHsDqfltbBDwAHNBPWw68uh8+ke4HxJeA44Cd+vZdgR364ecD5/TDx9DdFGzeYI3AM/tt+FF9+yMHtv83AP+zHz4feFY/PHdsHb42/+Xhke1YVd2b5KnAHwHPBT7fP7L2niTvAh5OF8RX0210AJ8ft5i/Ai6pqmMH2l7e36J3B7qjAIuBa4B/A/4uyVfoNvgx51dVJVkF3F5VqwCSXE33h2dhv4zv9AcLHgZcvBW+AmmmGjzk/gzgzCRPHGK+c4FTq+qz/XiA9yd5DvAg3bMx9uqn3VhVV/bDl9Nta1TVe5N8FngB8ErgFXQhvRuwLMl+dLfkfujAei+oqp8NjD+B7r++vaCqxu72uZDub8zedNvwjX37d4AP9+v8YlWtGeJzagIect/OVXdI7aKq+mvgLcCr6PbWX1ZVfwD8LbDTwCy/GLeIy4CnJpkHkGRf4J3AYVX1JOArdL/wH6B7mt45wJHA1waWsb5/f3BgeGx8B7o/ShdU1QH9a3FVvX4LP7o0K1TVxXR71/Pp9qoH/27vNK77d4Ajxk6T0W3P8+n28A8Abh+YZ3Bb28DANVVV9aOq+gRwGPDkJL8DvA+4sKqeCLyEjf9duI3uB/xTBto+Cnys/7ty3Nj8VXUK3R77zsD3kjx+0i9DG2Wgb8eSPK7/tT3mALrD4gDr+vPUm7qK/WvAKcBX+nNfu9Jt3Hf158iO6Nc1F9ituhsHva1f17C+Bzwrye/3y3p4kv+wGfNLs1YfcHOAnwI/BhYn2THJbnSBO+jEvt9p/fhuwB1V9askz2WSh3qMW9+LBn4Q7EcX9nf2y/pJ337MJhZzJ/AiuqMDhw7UMjb/0oH1PbaqVlXVB4GVgIE+RR5y377NBT6aZHe6X/6r6Z5kdyewiu6c3GWbWkhVfaEP8/OA/wRcQXeY/ga6PQaAXYBzk+xEt8f934ctsqrW9hf+fC7Jjn3z/6A79y+1aOckV/bDAZZW1QbgliTLgR8A19Nta+O9Dfh0klPprm85P8lKfn1OflNeA/xNkvvo/i68qqo29MtbluTtwDc3tZCquj3JS4CvJnkdcBLwhSQ/ofuRvu9Yvf2PjQ10p+a+OkSNmoB3ipMkqQEecpckqQEGuiRJDTDQJUlqgIEuSVIDDHRJkhpgoEsiyb2b0fekJO8c1fIlTY2BLklSAwx0SROa7OlYvScn+WaS65O8cWCeP0/3pL4fZOBJe5JGz0CXNJlvAwdX1VPonoj3roFpT6K7teczgBOT/F6SF9DdKvQgulv7PrV/KIikaeCtXyVNZrKnYwGcW1X3A/cnuZAuxJ9N94SusduRzqUL+G9NX8nS9stAlzSZjwIfrqrz+gdsnDQwbfw9o4vunuMfqKpPTUt1kn6Dh9wlTWbCp2P1liTZqX+s5qF0D/H5OvC6/sl6JFmQZM/pKlba3rmHLgng4UnWDIx/mMmfjgVwKd2z7h8FvK+qbgVuTfIE4OL+6Zv3Aq8G7hh9+ZJ82pokSQ3wkLskSQ0w0CVJaoCBLklSAwx0SZIaYKBLktQAA12SpAYY6JIkNcBAlySpAf8fKg16OhVtqbEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 576x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = newData.iloc[:]\n",
    "y= DataTweet[\"Label\"]\n",
    "#print('\\nK - Fold Cross Validation')\n",
    "k=10\n",
    "kf = KFold(n_splits=k)\n",
    "print(\"fold-berjumlah:\",k)\n",
    "kfold=[]\n",
    "temp_akurasi = []\n",
    "temp_pres = []\n",
    "temp_recall = []\n",
    "temp_f1 = []\n",
    "temp_model=[]\n",
    "it=0\n",
    "for train_index , test_index in kf.split(x):\n",
    "    X_train , X_test = x.iloc[train_index,:],x.iloc[test_index,:]\n",
    "    y_train , y_test = y[train_index] , y[test_index]\n",
    "   \n",
    "    sm = SMOTE(sampling_strategy=\"minority\",k_neighbors=5)\n",
    "    x_oversample, y_oversample = sm.fit_resample(X_train, y_train)\n",
    "    \n",
    "    if(it==1):\n",
    "        #setelah resampling dengan SMOTE\n",
    "        jumlah_awal = y_train.shape[0]\n",
    "        cekLabel_awal = Counter(y_train)\n",
    "        jumlah_sm = y_oversample.shape[0]\n",
    "        cekLabel_sm = Counter(y_oversample)\n",
    "        jumlah_tes = y_test.shape[0]\n",
    "        print('Jumlah Data latih sebelum SMOTE =', jumlah_awal)\n",
    "        print('Sarkasme =', cekLabel_awal['Sarkasme'],'BukanSarkas =',cekLabel_awal['BukanSarkas'])\n",
    "        print('Jumlah Data latih setelah SMOTE =', jumlah_sm)\n",
    "        print('Sarkasme =', cekLabel_sm['Sarkasme'],'BukanSarkas =',cekLabel_sm['BukanSarkas'])\n",
    "        print('Jumlah Data Uji =', jumlah_tes)\n",
    "        # creating the dataset\n",
    "        data = {'Sarkasme':cekLabel_sm['Sarkasme'], 'BukanSarkas':cekLabel_sm['BukanSarkas']}\n",
    "        courses = list(data.keys())\n",
    "        values = list(data.values())\n",
    "        fig = plt.figure(figsize = (8, 5))\n",
    "         \n",
    "        # creating the bar plot\n",
    "        plt.bar(courses, values, color ={'orange','blue'} ,width = 0.7)\n",
    "        plt.xlabel(\"Label\")\n",
    "        plt.ylabel(\"Jumlah\")\n",
    "        plt.title(\"Data Setelah SMOTE\")\n",
    "        plt.show()\n",
    "    it=it+1\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***Sebelum dilakukan SMOTE***"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfQAAAFNCAYAAAD2E503AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAkeklEQVR4nO3de7hcdX3v8fcHAgElgEiSw9UgRWqg3IyAYI+0VECF4lMvJ14qVi4elIO3tqJWRC0WtQc8alVQERQBwUtFkMpFkOIFDIogoIKCJkAhoNzkVsL3/LF+G4bNTrITmOxk5f16nnlm5rcu853Zs+Yzv99ae1aqCkmStGJbZaILkCRJT5yBLklSDxjokiT1gIEuSVIPGOiSJPWAgS5JUg8Y6NIYkhyR5KSJrmNpJTkhyT9PdB2jJXl9kosnuo5hSFJJ/mSi69DKy0BfiSW5Icl9Se5OckeSHyT530nG9b5IMqN9iE0acp2LfZwk6yY5Psl/tefzqyTvHGZdK6MkuyV5OMk97XJjkvdPdF2LMlDrPa32+wbuv2Yhy+yWZN6TWMOFSe5v7827klyW5LAkk5dgHRP2hSHJW5Jcn+SPSa5J8qyBaa9O8ts27d+TrDcwbXLbLu9q2+bbJ6L+lYWBrn2qagrwDOAo4J3A5ye2pKVyDLAW8GxgHeCvgV8vzYqG/QWlB26qqrWqai3g+cD+SV46wTUt1Eitrd7f0b3nR9q+vAxLOaRtaxsA7wBmA99OkmVYwxJLcgCwP/ASum1sb+C2Nm0r4Fjgb4HpwL3ApwYWPwLYgu7z5S+Af0yy17KqfWVjoAuAqrqzqs4A/hewX5KtAZK8JMlP2zfsuUmOGFjsonZ9R+vtPC/J5km+m+T2JLcl+XKSdUcWSPLO1qu7O8kvk+ze2ldpPZZft2VPG/im/7jHGeMpPBc4uar+UFUPV9UvquqrA4/7/1r9I72jPx+YdkSSryY5KcldwOsHV5xktSSnJPlaktWT/F3rpdyd5DdJ3jgw7/pJzmwjHr9P8p8jIx5tROQfklzRejOfTzI9ydltXecledrAunZuoyZ3JPlZkt0W9vdLsn2Sn7T1fAVYY2Da01pN85P8od3eeGD6hUk+mOT7bflzkqy/sMcaVFXXAz8AZrZ1PW40pa3/gIXU/dEkFydZZ1HvtYH17pfkd+299Z7x1Lgwrff4sSQ3tcvHWttTgbOBDfNoT37DJDsm+WH7e9yc5JNJVl/Sx62qP1bVhXRfOp9HF5Qsav1JRraBn7V6/tfi/q5PhvbefR/wtqq6ujq/rqrft1leA3yrqi6qqnuA9wJ/k2RKm/464INtu7wG+Cyjti89eQx0PUZVXQrMA0YC7490G+W6dB88B+fR3tj/bNfrtt7OD4EA/wJsSNdb3oTuWzpJtgQOAZ7beip7Aje0dRwKvBR4QVv2D8C/LeJxRvsRcGQL2y3GmP5jYDtgPeBk4PQkawxM3xf4anuej/TakqwJ/DvwAPDKqnoQuJWul7I28HfAMUl2aIu8g+71m0rXY3k3MPj7yi8DXgg8C9iHLjjeDaxPtz0e2h53I+As4J9bzX8PfC3J1NFPrH3o/zvwpTbv6e1xRqwCfIGul7QpcB/wyVGreXV7LtOA1dvjLVZ7rXele/3HLd0XuM8C2wB7VNWdLPq9NuL5wJbA7sDhSZ69JI87ynuAneneF9sCOwL/VFV/BF7EwEhEVd0ELADeRve3el6r4U1L++BV9TtgDo9uawtdf1WNbAPbtnq+wvj+ro8Y+KI51uXMhSy2cbts3b5kXZ/k/Xl0t9xWwM8GntOvgQeBZ7UvpxsOTm+3t1r8q6OlYaBrLDfRBQNVdWFVXdl6vVcAp9CF7piq6rqqOreqHqiq+cDRA/MvACYDM5OsVlU3tA8AgDcC76mqeVX1AN2XgJdn/MPf/4cuiA8Brk5yXZIXDdR1UlXdXlUPVdX/bXVsObD8D6vq39vzvK+1rQ38B93Q/d9V1YK2rrNaL6Wq6nvAOTz6ofzfdEOqz6iq/66q/6zHnjDhE1V1S1XdCPwncElV/bQ9528A27f5Xgt8u6q+3Wo6l+7D/8VjPPedgdWAj7XH/CrdF5iR5357VX2tqu6tqruBI3n83/ALVfWr9txPowu5hdmwhcBdwK+AS4AlOdBtNbr30Xp0w9/3tjrH8157f1XdV1U/owuHbZfgcUd7DfCBqrq1vVffTzd0PKaquqyqftTeQzfQDTUvdFsYp8FtbYnWP86/6+D8e1fVugu57L2QxUZ6/HsAf0Y3bP4quiF46Ibg7xy1zJ3AlDaNUdNHpmkIDHSNZSPg9wBJdkpyQRvWuxP433Q9iDElmZbk1HTD6ncBJ43MX1XXAW+lC+tb23wbtkWfAXxjpMcAXEP3BWD6eApuH/IfqqrnAE+nC6XT04btk7wj3TD5nW3964x6HnPHWO3OdD3IowZDOcmLkvwo3ZD6HXQhO7KujwLXAeekG44/bNQ6bxm4fd8Y90c+BJ8BvGKwF0XXO91gjDo3BG4c9cXhtwP1PiXJsekOXLqLbhfGuklWHZj/vwZu3ztQx1huaiGwNl1v+j7gxEXMP9qf0I2IvL+NeIzUOZ732pLUuTgbMvA6tdsbLmRekjyr9XL/q72OHxqjviU1uK0t0frH+Xd9oka+3H6kqu4Y+KIx8sXyHrovvoPWBu5u0xg1fWSahsBA12MkeS7dh8xIj+tk4Axgk6paB/gM3bA6PHYoecS/tPZt2gf+awfmp6pOrqrn0wVWAR9uk+YCLxrVa1ij9WSX6JSAVTXyYfhUYLN0+8vfCbwSeFpVrUvXUxg8GGmsxzinPZ/zk0yHbr8r8DXgX4HpbV3fHllXVd1dVe+oqmfSDam/Pe04gSU0F/jSqNfjqVV11Bjz3gxslDzm4KpNB26/g240Yqf2NxkZvn3CB2O1ofKT6Z4rdMPmAE8ZmO1/jFrsGrrh/bPbbpgRi3qvDcNNdO/DEZu2Nhj7/fBp4BfAFu11fPcTqS/JJsBz6EZqlmb9S/R3TXesxj0LuZy9kMf4Jd0Q+sK2wasYGCVJ8ky60a9fVdUf6N6bg6Mo27ZlNAQGugBIsnaSvYFTgZOq6so2aQrw+6q6P8mOdPtaR8wHHgaeOdA2he6b+R1tP/A/DDzGlkn+soXi/XTf/he0yZ+h2wf+jDbv1CT7LuJxRtf/3iTPTXfQ2hrAW4A76D6QpgAPtfVMSnI4j+9VjKmqPkIXNOenO1BsdboPrPnAQ21Yf4+BOvZO8ictXO9qz2/B49e8WCcB+yTZM8mqSdZI969UYx309MP2/A5NMinJ39DtDx4xhe61vqONWLxvKeoZU5K16I7WvgqgDV3fCLy21f0GYPPRy1XVKXSBdV6SkemLeq8NwynAP7X32vrA4XSvO3QjJ09Pss7A/FPo/qb3JPlT4OCledDWs34B8E3gUrovhONZ/y08flsb99+1ql5UA0f8j7q8aCHL3At8he7o9Cnt/XcgMLLP/ct079M/T3cw4QeAr7ddAABfpHuNn9ae04HACYuqU0vPQNe3ktxN1yN8D90+778bmP4m4ANtnsPphrKBRzb2I4Hvt2Hhnen2Q+5A1wM+C/j6wLom0/1r3G10Q6fT6D7UAf4fXe/snPZYPwJ2WsTjjFZ0BwjdRtfLeiHwkuqOvP0O3cFnv6IbVr2fsYfYx1RVH6Q76Ow8uv2/h7bX4Q90oXPGwOxbtPnuoQvaT1V3RPMSqaq5dMPS76b78jCX7svR47bZNmz9N3RHD/+B7j8VBl/3jwFr0r02P6I7LuCJeOTob7rXcz26/dEjDmy13k53ANQPxlpJVZ1IFwDfTTKDRbzXFifJa5Isac/vn+mOS7gCuBL4SWujqn5BF/i/ae+5DekOFHw13ZDxZ+mCbkl8sj23W+j+Jl8D9qqqh9v0xa3/CODEVs8refL/rgtzCN37+Sa69/TJwPEAVXUV3a6RL9MdLDqFxx4o+D66Y1B+C3wP+GhVDavOlV4eu9tNkiStiOyhS5LUAwa6JEk9YKBLktQDBrokST1goEuS1AMr9Fml1l9//ZoxY8ZElyFJ0jJz2WWX3VZVjzuvwwod6DNmzGDOnDkTXYYkSctMkt+O1e6QuyRJPWCgS5LUAwa6JGlM999/P1tuuSVJOOSQQwA4/vjj2XzzzVlzzTXZc889ufHGGx+Zf6eddmLKlCk85SlPYdasWVx00UUTVfpKaaiBnuSGJFcmuTzJnNa2XpJzk1zbrp82MP+70p3H+pdJ9hxmbZKkRfvABz7AvHnzHrk/Z84cDjjgADbaaCM+/OEPc+GFF3LwwY+eQ2aXXXbh4x//OO9973u5/PLLOeCAAyai7JXWsuih/0VVbVdVs9r9w4Dzq2oL4Px2nyQz6c7atBWwF/CpJ/m8vpKkcbriiis45phjOOKIIx5p+973vkdV8cY3vpFDDz2UHXbYgTPPPJPbb78dgKOPPpp99tmH3XffncmTJ7PKKg4CL0sT8WrvC5zYbp8IvHSg/dSqeqCqrgeu47GngJQkLQMPP/wwBxxwAG9+85t57nOf+0j7tGnTALj44ov5xS9+wbXXXktVccMNNwBw5513MnXqVHbaaSdWX311Pve5z01E+SutYQd60Z0O87IkB7W26VV1M0C7ntbaN+Kxp7Sc19okScvQF77wBW644QZe97rXPbKP/M4772SPPfZg11135TOf+QzPfvazefDBBwFYY401AFhrrbU455xz+PjHP87999/P4YcfPmHPYWU07P9D37WqbkoyDTg3yS8WMW/GaHvcuV3bF4ODADbddNMnp0pJ0iPmzp3L/Pnz2XbbbR9pO+mkk5g8eTIXXXQRV155JZMmTeKtb30rF198Mc985jMBmDRpEi984Qt54QtfyFe/+lUuuOACbrvtNtZff/2JeiorlaEGelXd1K5vTfINuiH0W5JsUFU3J9kAuLXNPg/YZGDxjYGbxljnccBxALNmzfJk7pL0JHvlK1/J1ltvDcBVV13FEUccwV577cXBBx/M2972Nrbffnt+/OMfc9555/H2t7+dNddck+985zucdtpp7LLLLsydO5cf/OAHTJ8+nac//ekT/GxWHkML9CRPBVapqrvb7T2ADwBnAPsBR7Xrb7ZFzgBOTnI0sCGwBXDpsOqTJI1t5syZzJw5E+CR3vXmm2/O9ttvz/7778+xxx7LU5/6VA455BA+9KEPAbDeeutxySWXcPLJJzN58mSe//zn85GPfIRkrMFXDUOqhtPJTfJM4Bvt7iTg5Ko6MsnTgdOATYHfAa+oqt+3Zd4DvAF4CHhrVZ29qMeYNWtW+dOvkqSVSZLLBv5z7BFD66FX1W+Abcdovx3YfSHLHAkcOayaJPWTnUAtr4bUZx6T/yQoSVIPGOiSJPWAgS5JUg8Y6JIk9YCBLklSDxjokiT1gIEuSVIPGOiSJPWAgS5JUg8Y6JIk9YCBLklSDxjokiT1gIEuSVIPGOiSJPWAgS5JUg8Y6JIk9YCBLklSDxjokiT1gIEuSVIPGOiSJPWAgS5JUg8Y6JIk9YCBLklSDxjokiT1gIEuSVIPGOiSJPWAgS5JUg8Y6JIk9YCBLklSDxjokiT1gIEuSVIPGOiSJPWAgS5JUg8Y6JIk9YCBLklSDxjokiT1gIEuSVIPGOiSJPWAgS5JUg8Y6JIk9YCBLklSDxjokiT1gIEuSVIPGOiSJPXA0AM9yapJfprkzHZ/vSTnJrm2XT9tYN53JbkuyS+T7Dns2iRJ6otl0UN/C3DNwP3DgPOragvg/HafJDOB2cBWwF7Ap5KsugzqkyRphTfUQE+yMfAS4HMDzfsCJ7bbJwIvHWg/taoeqKrrgeuAHYdZnyRJfTHsHvrHgH8EHh5om15VNwO062mtfSNg7sB881qbJElajKEFepK9gVur6rLxLjJGW42x3oOSzEkyZ/78+U+oRkmS+mKYPfRdgb9OcgNwKvCXSU4CbkmyAUC7vrXNPw/YZGD5jYGbRq+0qo6rqllVNWvq1KlDLF+SpBXH0AK9qt5VVRtX1Qy6g92+W1WvBc4A9muz7Qd8s90+A5idZHKSzYAtgEuHVZ8kSX0yaQIe8yjgtCT7A78DXgFQVVclOQ24GngIeHNVLZiA+iRJWuGk6nG7qVcYs2bNqjlz5kx0GZImWMY6AkdaDgwjYpNcVlWzRrf7S3GSJPWAgS5JUg8Y6JIk9YCBLklSDxjokiT1gIEuSVIPGOiSJPWAgS5JUg8Y6JIk9YCBLklSDxjokiT1gIEuSVIPGOiSJPWAgS5JUg8Y6JIk9YCBLklSDxjokiT1gIEuSVIPGOiSJPWAgS5JUg8Y6JIk9YCBLklSDxjokiT1gIEuSVIPGOiSJPWAgS5JUg8Y6JIk9YCBLklSDxjokiT1gIEuSVIPGOiSJPWAgS5JUg8Y6JIk9YCBLklSDxjokiT1gIEuSVIPGOiSJPWAgS5JUg8Y6JIk9YCBLklSDxjokiT1gIEuSVIPGOiSJPXA0AI9yRpJLk3ysyRXJXl/a18vyblJrm3XTxtY5l1JrkvyyyR7Dqs2SZL6Zpg99AeAv6yqbYHtgL2S7AwcBpxfVVsA57f7JJkJzAa2AvYCPpVk1SHWJ0lSbwwt0KtzT7u7WrsUsC9wYms/EXhpu70vcGpVPVBV1wPXATsOqz5JkvpkqPvQk6ya5HLgVuDcqroEmF5VNwO062lt9o2AuQOLz2ttkiRpMYYa6FW1oKq2AzYGdkyy9SJmz1ireNxMyUFJ5iSZM3/+/CepUkmSVmzL5Cj3qroDuJBu3/gtSTYAaNe3ttnmAZsMLLYxcNMY6zquqmZV1aypU6cOs2xJklYYwzzKfWqSddvtNYG/An4BnAHs12bbD/hmu30GMDvJ5CSbAVsAlw6rPkmS+mTSENe9AXBiO1J9FeC0qjozyQ+B05LsD/wOeAVAVV2V5DTgauAh4M1VtWCI9UmS1Bupetxu6hXGrFmzas6cORNdhqQJlrGOwJGWA8OI2CSXVdWs0e3+UpwkST0wriH3JLsAMwbnr6ovDqkmSZK0hBYb6Em+BGwOXA6M7NMuwECXJGk5MZ4e+ixgZq3IO9slSeq58exD/znwP4ZdiCRJWnoL7aEn+Rbd0PoU4Ookl9KdcAWAqvrr4ZcnSZLGY1FD7v+6zKqQJElPyEIDvaq+tywLkSRJS2+x+9CT7Jzkx0nuSfJgkgVJ7loWxUmSpPEZz0FxnwReBVwLrAkc0NokSdJyYlw/LFNV1yVZtf22+heS/GDIdUmSpCUwnkC/N8nqwOVJPgLcDDx1uGVJkqQlMZ4h978FVgUOAf5Id87ylw2zKEmStGQW20Ovqt+2m/cB7x9uOZIkaWks6odlrqT7YZkxVdU2Q6lIkiQtsUX10PdeZlVIkqQnZFE/LPPbwftJ1l7U/JIkaeKM5/SpbwQ+QLcPfWQIvoBnDrEuSZK0BMbT4/57YKuqum3YxUiSpKUznn9b+zVw77ALkSRJS288PfR3AT9IcgmPPX3qoUOrSpIkLZHxBPqxwHeBK4GHh1uOJElaGuMJ9Ieq6u1Dr0SSJC218exDvyDJQUk2SLLeyGXolUmSpHEbTw/91e36XQNt/tuaJEnLkfH8lvtmy6IQSZK09MbzwzKvG6u9qr745JcjSZKWxniG3J87cHsNYHfgJ4CBLknScmI8Q+7/Z/B+knWALw2tIkmStMTGc5T7aPcCWzzZhUiSpKU3nn3o3+LRk7KsCswEThtmUZIkacmMZx/6v/JooD8E/LaqbhxeSZIkaUktNNCT3E0X5Bk1qZI8QHfSlvdU1flDrE+SJI3DQgO9qqYsbFqSVYGtgS+3a0mSNIGW5qA4qmpBVf0M+MSTXI8kSVoKSxXoI6rq2CerEEmStPSeUKBLkqTlg4EuSVIPGOiSJPWAgS5JUg8Y6JIk9YCBLklSDxjokiT1gIEuSVIPDC3Qk2yS5IIk1yS5KslbWvt6Sc5Ncm27ftrAMu9Kcl2SXybZc1i1SZLUN8PsoT8EvKOqng3sDLw5yUzgMOD8qtoCOL/dp02bDWwF7AV8qv1mvCRJWoyhBXpV3VxVP2m37wauATYC9gVObLOdCLy03d4XOLWqHqiq64HrgB2HVZ8kSX2yTPahJ5kBbA9cAkyvqpuhC31gWpttI2DuwGLzWpskSVqMoQd6krWArwFvraq7FjXrGG01xvoOSjInyZz58+c/WWVKkrRCG2qgJ1mNLsy/XFVfb823JNmgTd8AuLW1zwM2GVh8Y+Cm0eusquOqalZVzZo6derwipckaQUyzKPcA3weuKaqjh6YdAawX7u9H/DNgfbZSSYn2QzYArh0WPVJktQnk4a47l2BvwWuTHJ5a3s3cBRwWpL9gd8BrwCoqquSnAZcTXeE/JurasEQ65MkqTeGFuhVdTFj7xcH2H0hyxwJHDmsmiRJ6it/KU6SpB4w0CVJ6gEDXZKkHjDQJUnqAQNdkqQeMNAlSeoBA12SpB4w0CVJ6gEDXZKkHjDQJUnqAQNdkqQeMNAlSeoBA12SpB4w0CVJ6gEDXZKkHjDQJUnqAQNdkqQeMNAlSeoBA12SpB4w0CVJ6gEDXZKkHjDQJUnqAQNdkqQeMNAlSeoBA12SpB4w0CVJ6gEDXZKkHjDQJUnqAQNdkqQeMNAlSeoBA12SpB4w0CVJ6gEDXZKkHjDQJUnqAQNdkqQeMNA1VIceeijTp08nCXvvvfcj7d///vfZZpttmDx5MjvssAM/+clPADjhhBNI8rjLDTfcMEHPQJJWDAa6hm727NmPuX///ffzspe9jLvvvptjjjmGW265hZe//OUsWLCAF7zgBZxyyimccsopfOlLX2L11Vdn+vTpbLTRRhNUvSStGAx0DdXHP/5x3va2tz2m7eyzz+aWW27hTW96E29605vYf//9uf7667nwwgvZbLPNmD17NrNnz2aNNdbgwQcf5A1veAOrrbbaBD0DSVoxGOha5q6//nqAR3rdG2+8MQC/+c1vHjPfscceyyqrrMJBBx20bAuUpBWQga4JV1UAJHmk7de//jXnn38+e+21FzNmzJigyiRpxWGga5nbbLPNAJg3bx4AN95442PaoeudVxUHH3zwsi9QklZAkya6APXbWWedxc9//nMA5s6dy+c+9zl22mknpk2bxqc//WmmTJnC5z//eWbMmMFuu+0GwIMPPsgJJ5zApptuyotf/OIJrF6SVhz20DVUH/3oRznssMMAuOKKKzjwwAO57LLLOP3001lrrbV4y1vewrRp0zj99NNZddVVAfj617/O/PnzOfDAA1llFd+ikjQeGdl/+aSvODke2Bu4taq2bm3rAV8BZgA3AK+sqj+0ae8C9gcWAIdW1XcW9xizZs2qOXPmDKV+SSuOgcMvpOXKMCI2yWVVNWt0+zCH3E8APgl8caDtMOD8qjoqyWHt/juTzARmA1sBGwLnJXlWVS0YYn1jO9lPBi2nXj2cL9+S+mFo45lVdRHw+1HN+wInttsnAi8daD+1qh6oquuB64Adh1WbJEl9s6x3UE6vqpsB2vW01r4RMHdgvnmtTZIkjcPycsTRWOPcY44vJjkoyZwkc+bPnz/ksiRJWjEs60C/JckGAO361tY+D9hkYL6NgZvGWkFVHVdVs6pq1tSpU4darCRJK4plHehnAPu12/sB3xxon51kcpLNgC2AS5dxbZIkrbCGdpR7klOA3YD1k8wD3gccBZyWZH/gd8ArAKrqqiSnAVcDDwFvnpAj3CVJWkENLdCr6lULmbT7QuY/EjhyWPVIktRny8tBcZIk6Qkw0CVJ6gEDXZKkHjDQJUnqAQNdkqQeMNAlSeoBA12SpB4w0CVJ6gEDXZKkHjDQJUnqAQNdkqQeMNAlSeoBA12SpB4w0CVJ6gEDXZKkHjDQJUnqAQNdkqQeMNAlSeoBA12SpB4w0CVJ6gEDXZKkHjDQJUnqAQNdkqQeMNAlSeoBA12SpB4w0CVJ6gEDXZKkHjDQJUnqAQNdkqQeMNAlSeoBA12SpB4w0CVJ6gEDXZKkHjDQJUnqAQNdkqQeMNAlSeoBA12SpB4w0CVJ6gEDXZKkHjDQJUnqAQNdkqQeMNAlSeoBA12SpB4w0CVJ6oHlLtCT7JXkl0muS3LYRNcjSdKKYLkK9CSrAv8GvAiYCbwqycyJrUqSpOXfchXowI7AdVX1m6p6EDgV2HeCa5Ikabm3vAX6RsDcgfvzWpskSVqESRNdwCgZo60eM0NyEHBQu3tPkl8OvSotrfWB2ya6iN54zVibhzQUbrtPkgxns33GWI3LW6DPAzYZuL8xcNPgDFV1HHDcsixKSyfJnKqaNdF1SFoybrsrpuVtyP3HwBZJNkuyOjAbOGOCa5Ikabm3XPXQq+qhJIcA3wFWBY6vqqsmuCxJkpZ7y1WgA1TVt4FvT3QdelK4a0RaMbntroBSVYufS5IkLdeWt33okiRpKRjoK7kk70lyVZIrklyeZKclWPaEJC8fZn3SyijJgrY9/izJT5LsMo5lbkiy/pPw2DsnuaQ9/jVJjljC5XdLcuYTrUNLbrnbh65lJ8nzgL2BHarqgfZhsPo4l/W9Iw3PfVW1HUCSPYF/AV6wjB77ROCVVfWz9nPcW453QT8XJpY99JXbBsBtVfUAQFXdVlU3JTk8yY+T/DzJcUn30whJLkzyoSTfA94yuKIkH2w99lWSfDrJnNbzf//APEclubqNBvxrazuhzX9Bkt8keUGS41vP4ISBZfdI8sPWWzk9yVrL4PWRlgdrA3+Ax/d+k3wyyesHZ06yZpL/SHJgkrWSnN+2myuT7NvmmdG2sc+27fScJGu2VUwDbgaoqgVVdXVbZsckP0jy03a9ZWt/fdsmvwWcM6qW57b5n5lkn9bz/2mS85JMb/O8oI0GXN6mTRnCa7hyqCovK+kFWAu4HPgV8CngBa19vYF5vgTs025fCHxqYNoJwMuBjwDH8uhBluu161XbMtsA6wG/HJhn3YF1nEr3K4H7AncBf0b3ZfMyYDu6X626CHhqW+adwOET/fp58TKsC7CgbZu/AO4EntPadwPOHJjvk8Dr2+0bgBnAecDrWtskYO12e33guratzQAeArZr004DXttuH073BeIbwBuBNVr72sCkdvuvgK+126+n+1Gw9QZrBHZp2/Cmrf1pA9v/AcD/bbe/Bezabq818hhelvzi8MhKrKruSfIc4M+BvwC+0k5Ze3eSfwSeQhfEV9FtdABfGbWa9wKXVNVBA22vbD/RO4luFGAmcDVwP/C5JGfRbfAjvlVVleRK4JaquhIgyVV0Hzwbt3V8vw0WrA788El4CaTl1eCQ+/OALybZehzLfRP4SFV9ud0P8KEk/xN4mO7cGNPbtOur6vJ2+zK6bY2q+kCSLwN7AK8GXkUX0usAJybZgu4nuVcbeNxzq+r3A/efTfevb3tU1civfW5M9xmzAd02fH1r/z5wdHvMr1fVvHE8T43BIfeVXHVDahdW1fuAQ4DX0PXWX15VfwZ8FlhjYJE/jlrFj4HnJFkPIMlmwN8Du1fVNsBZdN/wH6I7m97XgJcC/zGwjgfa9cMDt0fuT6L7UDq3qrZrl5lVtf8TfOrSCqGqfkjXu55K16se/NxeY9Ts3wdeNLKbjG57nkrXw98OuGVgmcFtbQEDx1RV1a+r6tPA7sC2SZ4OfBC4oKq2BvZh0Z8LN9N9gd9+oO0TwCfb58obR5avqqPoeuxrAj9K8qcLfTG0SAb6SizJlu3b9ojt6IbFAW5r+6kXdxT7fwBHAWe1fV9r023cd7Z9ZC9qj7UWsE51Pxz01vZY4/UjYNckf9LW9ZQkz1qC5aUVVgu4VYHbgd8CM5NMTrIOXeAOOrzN96l2fx3g1qr67yR/wUJO6jHq8V4y8IVgC7qwv6Ot68bW/vrFrOYO4CV0owO7DdQysvx+A4+3eVVdWVUfBuYABvpScsh95bYW8Ikk69J987+O7kx2dwBX0u2T+/HiVlJVp7cwPwN4MfBTumH639D1GACmAN9MsgZdj/tt4y2yqua3A39OSTK5Nf8T3b5/qY/WTHJ5ux1gv6paAMxNchpwBXAt3bY22luB45N8hO74lm8lmcOj++QX52+BY5LcS/e58JqqWtDWd2KStwPfXdxKquqWJPsAZyd5A3AEcHqSG+m+pG82Um/7srGAbtfc2eOoUWPwl+IkSeoBh9wlSeoBA12SpB4w0CVJ6gEDXZKkHjDQJUnqAQNdEknuWYJ5j0jy98Nav6SlY6BLktQDBrqkMS3s7FjNtkm+m+TaJAcOLPMP6c7Ud0UGzrQnafgMdEkLczGwc1VtT3dGvH8cmLYN3U97Pg84PMmGSfag+6nQHel+2vc57aQgkpYBf/pV0sIs7OxYAN+sqvuA+5JcQBfiz6c7Q9fIz5GuRRfwFy27kqWVl4EuaWE+ARxdVWe0E2wcMTBt9G9GF91vjv9LVR27TKqT9BgOuUtamDHPjtXsm2SNdlrN3ehO4vMd4A3tzHok2SjJtGVVrLSys4cuCeApSeYN3D+ahZ8dC+BSunPdbwp8sKpuAm5K8mzgh+3sm/cArwVuHX75kjzbmiRJPeCQuyRJPWCgS5LUAwa6JEk9YKBLktQDBrokST1goEuS1AMGuiRJPWCgS5LUA/8fSC2tgeHw7LAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 576x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "jumlahData=DataTweet[\"Tweet\"].shape[0] #Jumlah Data\n",
    "jmlSarkas=DataTweet[DataTweet.Label==\"Sarkasme\"].shape[0]\n",
    "jmlNonSarkas=DataTweet[DataTweet.Label==\"BukanSarkas\"].shape[0]\n",
    "d={'Jumlah': [jumlahData], 'Sarkasme': [jmlSarkas],'BukanSarkas':[jmlNonSarkas]}\n",
    "\n",
    "cekJumlah = pd.DataFrame(data=d, columns=['Sarkasme','BukanSarkas'])\n",
    "# creating the dataset\n",
    "data = {'Sarkasme':jmlSarkas, 'BukanSarkas':jmlNonSarkas}\n",
    "courses = list(data.keys())\n",
    "values = list(data.values())\n",
    "fig = plt.figure(figsize = (8, 5))\n",
    "\n",
    "# creating the bar plot\n",
    "plt.bar(courses, values, color ={'orange','blue'} ,width = 0.7)\n",
    "plt.xlabel(\"Label\")\n",
    "plt.ylabel(\"Jumlah\")\n",
    "plt.title(\"Dataset Sarkasme dan Bukan. Total Data = %s\" %jumlahData)\n",
    "for x, y in enumerate(values):\n",
    "    plt.text(x , y + 2, str(y),color = 'black', fontweight = 'bold')\n",
    "plt.show()"
   ]
  },
  {
   "attachments": {
    "kfold10.png": {
     "image/png": "iVBORw0KGgoAAAANSUhEUgAAAe4AAAFCCAIAAAC4q6mUAAAAAXNSR0IArs4c6QAAAARnQU1BAACxjwv8YQUAAAAJcEhZcwAADsMAAA7DAcdvqGQAAClUSURBVHhe7Z1PjBxFvueLaXgYjLENxgx/3lM/ibeaHeG3O+DliV4ttHYvluCxli+LJZBalihGGo1smZE4cLA4+djFbQ4jceG4mpbeSCXNyYcRGq3mwJFDHfo2HDn6yEZERlZGRmZU5S8yMlxZ9fnoK5EZGRlRGZX5qajo8szkJ8N33313DwKowSlGScTp6ak9Pxe3bt2yW1n44osvPvnkE7uThdsGu5OFzEOqyNxj/iFV94y6c+xOFrZ+SO/fv//w4UOr8qOjo+vXr9sj4KCGRQ1OMUoi1LkHlx/77PV8OXvu3M07d5Rc8+Tta9deePllr3DQvLS//+bhoVc4aDIP6Qe3bj119qxXOGjyD+lzly+/f3TkFQ6XXRjSV/b3Hzx4UKn866+/LrbBRQ1LtMqVXv/+4c+y5Zlnz/3H6enffvopT+4eH//y6lWvcNC88e676sb1CgfNM+fP5xzSP/zlLxdfeMErHDT5h/TFV1/9/YMHXuFw2YUhfevwEJWvB5WHgsqTB5UnDyoHCyoPBZUnDypPHlQOFlQeCipPHlSePKgcLKg8FFSePKg8eVA5WFB5KKg8eVB58qBysKDyUFB58qDy5EHlYEHloaDy5EHlyYPKwYLKQ0HlyYPKkweVgwWVh4LKkweVJw8qX8tidjBp42C2sDXamE8nk+nc7tQw7TUOhesLME2vfl1hUHkoqDx5UHnyoPJuaNPWHLmYTcPK1LUVLWq2R7xD9tOin8rLpsej8m9e06/3mw/3vPK1Eat8MbtixkYznftH1yVG5W6PB7MT7+i69HhI5jdMl3cXXvmaSFX+lb3bLFdmC6/C6kR6p8eoiofU7atEdJlylRfvXcH0K//omkQNqdPjwHepuWG8i1rcrWbJna53EJWvoX3qbWg9tKK+BN3MOFS+95F9CzOoXN2v5Y1ink+pd+QqV/foUqb6aZH2GK3yE30DHFzJo3L5h+IyMd4x792NeaO8W/pOIXXvslEVqtx4rRxSPbxCt8qH1L0za713TOchXX5g1HztXmPH602u8sVstk66qHxl9Hz80t637+i3N8es3EmEg3ousET0GOmdQjdzsXRUNl/lPXvsqfKI3oUq176rPqi0cGQTc/GQel3Ie+w4pHrolKa99r2Pxm6flMlVPp/WpGs0XODY3lOzbsBhhcqrmrZS1f5aS49F5Ta7oPKouWScd9Sl6UmWfP6osvEqr5tOnl4qj3oTY2bldmYaM0fuq3L5bSMbUq87b7fb+5tK5Q6ViV15GucWO46aFfrsslLR0kqVuzJWB2rN+yfW0VXcsyXshMqjnsmea+URAorxju7RPBvyZ1Kl11q5cClARewdc1E3puoJsGRbs1Ixy1bia5SvlRuDR12dStSQOh0ZSWVTuR7Smsr1ta+96iFn5eb1VXZd2tbVrufXViPbQq+5UvwOq0XtdSViB1QeM9lR6bfAYhYKhZ3KveM8CVlU7sSMqtB0m+6dWiK/EIhVbq5xMp0VQpf2KP+iUwxjifxPLFug8mqtvKHlsp5zwK/TOEdjCgt8lbeo2X0HatV1M6g8ELtm1yhfm55r5d5N3CVi75j7xF5abpXHiDVO5Y7dxHqNVnlDOl0jU7l3gRmG1MvQd6nXvt9d1gWWNkfqctem5gWqXdfX3rlhlatC81+nctleR1B5KGZBIOaBVNl8ldeWO5ZIvgr0VrnsAuXe0Y+6M2vLpnKvX0FkKvfHUHyBPVUe8cePXir3Jhzd5h9Dqrxwb2nbyqSlmg365GqvdqjEKTS1l8fNgWXP9XX0JsGX2YEtVnkfj6uIVa7uy2r6H7OqEz2F1On2VHiRqVzdZ9UVaelILzDCO+6XqoiZcuSQmidKOphF+szKzQVmnJVHXWYvldefi44fJD1VbmS6xHewwqlgNapftbvvlFRULVUN6PpOc47bC1p6X+J2EaPznCov/nGQy5fvCYQuU7l5SOrI7tqIWXltmizUnMqmq7x4DksiJq1R3nE7FX8wRw1pzMfwMuK18pokxO9g3BedEnF3Kh2HtPYsaJZ9OS+g28pniln5DvAIZuWxif2zZ2T6LrDI00vlUem1wCJPz9WAiOQfUrHK+2UXhhSVdwKVh4LKkweVJw8qBwsqDwWVJw8qTx5UDhZUHgoqTx5UnjyoHCyoPBRUnjyoPHlQOVhQeSioPHlQefKgcrCg8lBQefKg8uRB5WBB5aGg8uRB5cmDysGCykNB5cmDypMHlYMFlYeCypMHlScPKgcLKg8FlScPKk+eXVT59evXlX3AQw1LtMoPLj+mbJ4tZ8+du3nnjrqN8uTta9deePllr3DQvLS//+bhoVc4aDIP6Qe3bj119qxXOGjyD+lzly+/f3TkFQ6XXRjSV/b3K5Wfnp4W5oImanCKURKRf0h/+9vf2q1c/OY3v7FbWbh9+/bnn39ud7KQ+QIVmXtU46lG1e5kgSFNzvHxsRKOVTkAAIwXVA4AMHpQOQDA6EHlAACjx6r8+PjY/v9WQIPirwpSvv76a3t+Li6efdJu5eLMU0/ZrS3lwvPP261cPHnmjN3aUp69eNFu5WLrh/T8hQunp6dW5fyuPESf35V/kvfnpeee3Pt//+7/2Hy4fPmrx3755pveaxg0/K48efhdefI8+t+Vo/JWUHkoqDx5UHnyoHKwoPJQUHnyoPLkQeVgQeWhoPLkQeXJg8rBgspDQeXJg8qTB5WDBZWHgsqTB5UnDyoHCyoPBZUnDypPHlQOFlQeCipPHlSePKgcLKg8FFSePKg8eVA5WFB5KKg8eVB58qDytSxmB/afjdY5mC1sjTbm08lkOrc7NUx7jUPh+l3RLRTEtYPKQ0HlyYPKkweVd0N7subuxWwaVrm1aotSS9/WD9lPix4qVy3Ys4u2IprKr/Kv9GBMv2qUr41Y5e/tXdUDbHhtzz+6LjEqX8yu2P70jXPiHV2XHg/J/Ibp8u7CK18TqcrNe1dxZbbwKqxOpHd6jKp4SN2+SkSXKVd58d4ViJ+LqCF1ehz4Lm172Bd3q1lyp+sdROVraJ96G1oPrajfhfnMeW2RbeVV+fIeyqDyvY8mk28+NAY3Tr/6hszmcpWre3QpU32lUtNFq/xEvfcHB1fyqHw69wq7J8Y7xq035o3ybuk7hdS9y0ZVqHLjtXJI9fAK3SofUvfOrPXeMZ2HtP1hd6+x4/UmV/liNlsnyswqr6Nf7Gar3L5z5oUOr/JavnlNPDHvucASYb1I7xS6mYulo7L5Ku/ZY0+VR/QuVLn2XfVBJX80xEPqdSHvseOQtj/s3kdjt0/K5CqfT2uiNBoucGzvqVk34LBC5VVNW6lqv9v3Al3fb78DeWflJvK7p8iYVB41l4zzjnpm9CRLPn9U2XiV100nTy+VR72JMbNyOzONmSP3Vbn8tpENqded/+x3en9TqdyhMqX2ZmlY49xix1GzQp9dVipa8lXr1J9PXWOrA7Xm1zvaOUPETqjcLLB89M7wKjcPf0GEgGK8o3s0g5lN5Us6fDX2IvaOuagbU/UEWLKtWamYZSvxNcrXyo3Bo65OJWpInY6MpLKpXA9p7dnX1772qoeclZvXV9l1aVtXu3rbkWurkW2h11wpfoe1lq5/EgjYAZXvfXkp1589q5iFQuEMS+4d50nIonIn7nSyazbdO7VEfiEQq9xc42Q6K4Qu7VH+RacYxhL5n1i2QOXVWnlDy2U954BfJ6zyAl/lLWZ234FaddWM33Jntl7lemnl0t63xd8/Jem5Vh5xmWLvmPvEyjS3yosLlPUYp3LHbmK9Rqu8IZ2ukancu8AMQ+pl6LvUa9/vLusCS9tsV5e7NjUvUO26vvbODatcFZr/OpXL9jrhTsg7/GnWY7tVrj2+/B2LMJuv8tpyxxLJV4HeKpddoNw7+lF3Zm3ZVO71K4hM5f4Yii+wp8oj/vjRS+XehKPb/GNIlRfuLW2rd4paNV/rk6u92qESp9DUXh43B5Y9r1g9KTqp8HtYyxarvI/HVcQqV/dlteBg1h8GX2Bx0u2p8CJTuXrjqivS0pFeYIR3tGvKUY2YKUcOqb5FxYNZpM+s3Fxgxll51GX2Unn9uej4QdJT5UamS1oM6VSwotWv2t1vilZRtVQ1oOs7zTluLwj4uf4SDWKTZ1V5YxYpu41kKnf/fVDJl+8JtB4xK69doFBzKpuu8uI5LImYtEZ5x+1UPAOIGtKYj+FlxGvlNUmI38G4Lzol4u5UOg5p+GF3XkC3v7WkmJXvAI9gVh6b2D97RqbvAos8vVQelV4LLPL0XA2ISP4hFau8X3ZhSFF5J1B5KKg8eVB58qBysKDyUFB58qDy5EHlYEHloaDy5EHlyYPKwYLKQ0HlyYPKkweVgwWVh4LKkweVJw8qBwsqDwWVJw8qTx5UDhZUHgoqTx5UnjyoHCyoPBRUnjyoPHlQOVhQeSioPHlQefKgcrCg8lBQefKg8uRB5WBB5aGg8uRB5cmzcypX3rH/Ay7QQA1OMUoi1GeAPT8Xz57Zs1u5ePyJJ+zWlnLu/Hm7lYsnz5yxW1uK+nS0W7nY+iFVF/j9999blQMAwHhB5QAAoweVAwCMHlQOADB6rMpPT0/vQQA1OMUoifjxxx/t+bn4/PPP7VYufve739mtLKgLzHyN+Yc0/wVm7jHzPaPY+iE9Pj5WwrEqPzo6un79uj0CDmpYon+M+Obh4SfmJ4l5cvbcuZt37niFw+Xta9d+/vTPPnv9sWz5x7OT7R7SD27deursWa9w0Ly0v595SJ+7fPn9oyOvcLjswpC+sr/P78rXM6LflWf+EfTd4+P/8sIT3o/NB83blyfbPaT8rjx5+CdCYEHloaDy5EHlyYPKwYLKQ0HlyYPKkweVgwWVh4LKkweVJw8qBwsqDwWVJw8qTx5UDhZUHgoqTx5UnjyoHCyoPBRUnjyoPHlQOVhQeSioPHlQefKgcrCg8lBQefKg8uRB5WBB5aGg8uRB5cmDyteymB3Y/+HzOgezha3Rxnw6mUzndqeGaa9xKFy/K7qFgrh2UHkoqDx5UHnyoPJuaE/W3L2YTcMqt1ZtUWrp2/oh+2nRQ+WqXXt2+ydFB/Kr/Cs9GNOvGuVrI/bOYnZFD7BhOvePrkuMyt/bu2r7m0wu7X374Z5fYWV6qHx+Q3d5cHfhla+JdEjNe1dxZbbwKqxOpHfc9/FgduIdXRmxd9y+SkSXKVd58d4ViJ+LqCF1ehSOp4poSNse9sXdapbc6XoHUfkaVgi19VC0gAsWC+elqdcqeakleVW+vIcyqFz1VfZink+pd+Qq3/vy0uTL9wp97300mVx9I5PKT9R9dHBwJY/K5R+Ky8R4x7x3N+aN8m7pO4XUvctGVahy47VySPXwCt0qH1L9DJbPQq33juk8pO0Pu3uNHa83ucoXs9k66WZWuYtqKqqhnCq375we1QwqryXCQT0XWL55bTJ5LYvKC93MxdJR2XyV9+yxp8ojeheqXPuu+qCSPxriIfW6kPfYcUjbH3bvo7HbJ2Vylc+nNVcaDRc4tvfUrBtwWKHyqqatVLW/frKt6sbMyDV5Z+Um8runyJhUblZaPnonh8rVpelJlnz+qLLxKq+bTp5eKtdDKu49ZlZuZ6Yxc+S+KpffNrIh9brzn/1O728qlTtUJtaeLeVpnFvsOGpW6LPLSkVLK1Xuythxc73NJuZ4wYpaQXZC5VHPZM+1cqnHVWJUri/NDGY2lS/p8NXYi9g75qJuTJd3uHiVrI/KzbKV+Brla+XV2rH06lSihtTpyEgqm8r1kNaefX3ta696yFm5eX2VN5e2dbWrtx07txrZFnrNleJ3cDXfimlpfbUmO6DymMmOSr8FFr1WPvwCi/MkZFG5EzOqQtNtundqifxCIFa5ucbJdFYIXdqj/ItOMYwl8j+xbIHKq7XyhpbLes4Bv07jHE0hYIOv8hYru++A31LRFipvxq7ZNcrXpuda+d/f0e/TN5IfsYhVbu4Te2m5VR4j1jiVO3YT6zVa5Q3pdI1M5d4FZhhSL/KHsY/KG91lXWBp06Mud21qXqDadX3tnRtWuSo0/3Uql+2JUCeJz9l2lZsFgZi+VDZf5bXljiWS7x+9VS4bW7l39KPuzNqyqdzrVxCZyv0xFF9gT5VH/PGjl8q9CUe3+ceQKi/cW5pT7xS1ar7WJ1d7tUMlTqGpvTxuDix7rq+jt6POWF+phS1WeR+Pq4hV/t7e1eq35PqHiZl+wVIkw6xcvXHVY6+l4+x2SoR33C9VETPlSJWbB186mEX6zMrNBWaclUddZi+V11c7O36Q9FS5kekS38EKp4J1qH7V7r5TUlG1VDWg6zvNOW4vaOnd4DYf5XFFTpU3ZpGy20jmHfOQ1JF1FzEr1z9AXCL0uMqmq7x4DksiJq1R3nE7FX8wR6m8phtpxGvlNUmI38G4Lzol4u5UOg5p+GF3XkC3lc8Us/Id4BHMymPTazVAnr4LLPL0UnlUMg9pz9WAiETOyntErPJ+2YUhReWdQOWhoPLkQeXJg8rBgspDQeXJg8qTB5WDBZWHgsqTB5UnDyoHCyoPBZUnDypPHlQOFlQeCipPHlSePKgcLKg8FFSePKg8eVA5WFB5KKg8eVB58qBysKDyUFB58qDy5EHlYEHloaDy5EHlyYPKwYLKQ0HlyYPKk2cXVX79+nVln03g9u3bn3/+ud151KhhiVb5m4eH6k3NlrPnzt28c8crHC5vX7v286d/9tnrj2XLP56dbPeQfnDr1lNnz3qFg+al/f3MQ/rc5cvvHx15hcNlF4b0lf39SuXfffddYa5N4F2D3dkA1OAUoyQi/5DevHnTbmXhiy+++Pjjj+1OFj799FP1MW93sqAuUF2m3Rme/EOqxlONqt3JAkOanPv37z98+NCqfKMoXp/dAQCAdaByAIDRg8oBAEYPKgcAGD2oHABg9KByAIDRg8oBAEYPKgcAGD2oHABg9KByAIDRg8oBAEYPKgcAGD2oHABg9KByAIDRg8oBAEYPKgcAGD2oHABg9KByAIDRg8oBAEYPKgcAGD2oHABg9KByAIDRg8oBAEYPKgcAGD2oHABg9KByAIDRg8oBAEYPKgcAGD2oHABg9KByAIDRs0Eqv3//fiHxdw3Ftiq0hwEAIMAGqfzOnTuTBqrQHgYAgAAbpPIffvjhzJkzVuEGtasK7WEAAAiwWWvl3sScKTkAQBc2S+XuxJwpOQBARzZL5YrlxJwpOQBARzZO5cXEnCk5AEB3Nk7lCjUfZ0oOANAdq/Lj4+NiWQOaqMEpRknEycmJPT8Xz7/4ot3KxVNPP223tpT8Q3r+4kW7taVcvHTJbuXiwtNP2q0t5fyFC6enp1blR0dHX3/9dbENLmpY1ODYHQn37t375N69v/30U7Y8c/78f5yeeoXD5e7x8S+vXvUKB80b77673UP6h7/85eILL3iFgyb/kL746qu/f/DAKxwuakifP/Ozv3+YL29fnmQe0rcODx88eIDK14DKQ0HlyYPKkweVgwWVh4LKkweVJw8qBwsqDwWVJw8qTx5UDhZUHgoqTx5UnjyoHCyoPBRUnjyoPHlQOVhQeSioPHlQefKgcrCg8lBQefKg8uRB5WBB5aGg8uRB5cmDysGCykNB5cmDypMHla9lMTuw/2y0zsFsYWu0MZ9OJtO53alh2mscCtfvjG6iwysLgcpDQeXJg8qTB5V3Q2uyZsjFbBoWppVqi5pL3dYP2U+LPip3Pgoar7Ub+VX+lR6M6VeN8rURe2cxu6IH2DCd+0fXJUblbo8HsxPv6Lr08M78huny7sIrXxPpkJr3ruLKbOFVWJ1IlfcYVfGQun2ViC5TrvLivSsQPxdRKt/7yHY3mVza+/bDvUaFVRGpvO1hX9ytZsmdrncQla+hfeptaD20on4X9OnO2Y7Xu5NX5cu7NoPKVV9lL+b5lHpHrnJ1jy5lqq9U2mO0yk/UjXBwcCWPyuUfisvEqNy8dzfmjfJu6Tsr173LRlWocuO1ckj18Ao/q+Qq1x6/+kah770vL00mrw2k8vaH3b3GjtebXOWL2WydJzOr3HN3VGs5VW7fOfOyh1d5LREO6rnAEtFjpHcK3czF0lHZfJX37LGnyiN6F6pc+676oJI/GmKVv6Pl+s1yJu7tdkhHleuhaz7s3kdjt0/K5CqfT2uaNOIscGzvyVQ34OB71qlf1bSVqvaD3wtqL28EKrfx3t3OGZPKo+aScd5Rl6an//L5o8rGq7xuOnl6qTzqTYyZlduZaW2G3jF9Vf7e3tXJ5Mv30qvcxnvY/We/0/ubSuUOlSa1NUuJGoMWO3WZuqYtWvI969SfT11jqwO15gOCNq0uz3P768xOqDzqmey5Vh4hoBjv6B7NYGZT+ZIOX429iFVuLurGVD0BlmxrVipm2Up8jfK18mrtWHp1KmKVG3eXCyzW7NlUroe09uzra1971UPOys3rq+y6tK2rXb3tiLXVyLbQa84q2iVoaNOCg9/DWnZA5TGTHZV+CyxmoVDYqdw7zpOQReVOzKgKTRelcudRN8+j6Bp7qDzyC4FY5eYaJ9NZIXRpj2KVqxh9Wy5Ncs7KN0Tl1Vp5Q8tlPeeAX6dxjsYRce2Qaq9F3rqXEr+loq2g8cNsvcrtml2jfG16rpVHXKbYO+Y+sZeWW+UxYo1TuWM3sV6jVd6QTtfIVO5dYNSQilXuZrC1chvvKfAfiqwLLG161OWuTc0LVLuur71zwypXhea/TuWyve5EinzbVW4WBGL6Utl8ldeWO5ZIvgr0VrnsAsUqN4+6M2vLpnKvX0FkKvfHUHyBPVX+zWvD/YLFxLtAb8LRbf4xpMoL95a2rTRaqtmgT672aodKnEJTe3ncHFj2XF9Hb2AqN9ruyBarvI/HVcQqV/dlNf2PWdXpsRrQ9anwIhtS9cZVV6SlI71AucprX6oiZsqRQ2oefOlgFukzKzcXmHFWLl8oV+ml8vpzod/cDrdQT5UXfixp8aRTwYrW2NjZd0oqqpaqBnR9pznH7QUhS5fnrBT9GnKqvDGLlN21Mu+Yh6SOrLuIWXntAoWaU9l0lRfPYUnEpDVC5fVOxR/MUUMa8zG8jHitvCYJ8TsoV7nz74PkHlfpqPLww24mAQXdVj5TzMp3gEcwK49Nr9UAefousMjTS+VRyTykUSrvlfxDKlZ5v/SalUdFNitPEVTeCVQeCipPHlSePKgcLKg8FFSePKg8eVA5WFB5KKg8eVB58qBysKDyUFB58qDy5EHlYEHloaDy5EHlyYPKwYLKQ0HlyYPKkweVgwWVh4LKkweVJw8qBwsqDwWVJw8qTx5UDhZUHgoqTx5UnjyoHCyoPBRUnjyoPHlQOVhQeSioPHlQefLsnMqVd+z/gAs0UINTjJII9Rlgz8/FufPn7VYuHn/iCbu1peQf0ifPnLFbW4r6dLRbuXjyiT27taWoe+b777+3KgcAgPGCygEARg8qBwAYPagcAGD0WJX/+OOPDyCAGpxilETkH9KTkxO7lYs//vGPdisLf/rTn/785z/bnSxkvkBF5h7VeKpRtTtZYEiT89e//lUJx6r817/+9S9+8YtDaKCGRQ1OMUoi7t2798r+/luHh9ly5umn//Xf/s0rHC77//IvT5096xUOmnMXLrz46qte4aB5+pln/vWtt7zC4fKff/Wrxx9/3CscNM9evJh5SJ85f/4/XbniFQ6XRzKkLz/z+H//54vZcv7pJ5XNrcr5XXkIflceCr8rT54/8Lvy1HkkQ/rZ6495PzYfNMrmam6OyteAykNB5cmDypMHlYMFlYeCypMHlScPKgcLKg8FlScPKk8eVA4WVB4KKk8eVJ48qBwsqDwUVJ48qDx5UDlYUHkoqDx5UHnyoHKwoPJQUHnyoPLkQeVgQeWhoPLkQeXJg8rBgspDQeXJg8qTB5W3spgd2P+x8zoHs4Wt0cZ8OplM53anhmmvcShcX4BpuvG6lhfQvX1UHgoqTx5UnjyoPIw2bc2Ri9k0rHJdu12d9oh3yMq2n8rLpj2VV58R1dZa8qv8K/3qp181ytdG7J3F7EoxUIrp3D+6LjEqd3s8mJ14R9elh3fmN0yXdxde+ZpIh9S8dxVXZguvwupEeqfHqIqH1O2rRHSZcpUX712B+LmIGlKnx6i7tLvKv3lNd/LNh3tO4d6Xl0zXhvqh9iRT+Rrap96G1kMr6kvQzXifOAdOs+oqul1EXpUv76EMKld9lb2Y51PqHbnKF3cPljLVVyrtMVrlJ/pmOLiSR+XyD8VlYrxj3rsb80Z5t/T4dDTRvctGVahydc9UQ6qHV+hW+ZC6d2at947prPK9j+zDXvO1lvulvW9Nibu9IklUvpjN1kl3Q1Tu7fuHg+RUub1TzXeGHLNyJxEO6rnAEtFjpHcK3czF0lHZfJX37LGnyiN6F6pci7X6oJI/GuIh9bqQ99hR5VbT79RV/t7e1cnky/cCu4EkUfl8WpOu0XCBY3tPzboBhxUqr2raSlX7azXsu9q8Jd6r8LtuI++s3MS8tC1XedRcMs476tL0JEs+f1TZeJXXTSdPL5VHvYkxs3I7E4+ZI/dVufy26ahyG0/l3q6ZuX/0zoAqd6h06MrTOLfYcdSs0GeXlYqWfJ869edTV8bqQK351SLWVdyzzTvknOHvh9gJlUc9kz3XyiMEFOMd3aMZTPkzqRKj8iXyZVaxd8xF3Ziqp8GSbc1KxSxbia9RvlZuDB51dSpRQ+p0ZISVTeXfvlHbLdbNr74xoMqXjnRm5Z4bl7Z1tev5tdXItrChWl1QwxV1E6+r4Mtbxw6oPGayo9JvgUVPJ6Wdyr2jL80+lllU7sSdTnbNI/FOrMojvxCIVW6ucTKdFUKX9ij/olMMY4n8TywjVXm1Vt5wY1nPOeDXafWpKSyoHVLttcjbHfVadd2MW9/b9w8H2XqV64mkfG6l0nOtPOIyxd4x94y9tNwqjxFrnModu4n1Gq1yPSWPuktlKvcuMMOQeom6S0e0wNImQV3u2tSMgdp1fe2d6x5aUhaa/zqVy/Y60nC1Llie3zgaZLtVbhYEYvpS2XyV15Y7lki+CvRWuewC5d7R7naWHbKp3OtXEJnK/TEUX2BPlUf88aOXyh/dnz1djHtLW1auNMWlQ/XJ1V7tUIlTaGovj5sDy57r6+hNWl6mKrIl+qDfb4AtVnkfj6uIVa5mWNX0P2ZVJ3oKqZNhVq7euOqKtHSkFxjhHfdLVcRMOXJIzdMlHcwifWbl5gIzzsqjLrOXyosflb9md/WvXMrtFYlQuZHpkhYXOhUcabr7TklF1VLVgK7vNOe4vWCVid0uajpfNtjR44qcKm/MImW3kcw75iGpI+suYlZeu0Ch5lQ2XeXOH+gUEZPWKO+4nYo/mKOGNOZjeBnxWnlNGOJ3MO6LTom4O5WOKi/+cZBLOfuufm/e5UflKrGz8h3jEczKY9NrNUCevgss8vRSeVQyD2nP1YCI5B9Sscr75ZEMqWBWniKovBOoPBRUnjyoPHlQOVhQeSioPHlQefKgcrCg8lBQefKg8uRB5WBB5aGg8uRB5cmDysGCykNB5cmDypMHlYMFlYeCypMHlScPKgcLKg8FlScPKk8eVA4WVB4KKk8eVJ48qBwsqDwUVJ48qDx5UDlYUHkoqDx5UHny7KLKr1+/ruwDHmpYolX+5uGhek6y5ey5czfv3PEKh8vb16698PLLXuGgeWl/f7uH9INbt546e9YrHDT5h/S5y5ffPzryCofLIxnSg8uPKZtnyz9dOFOp/LvvvivMBU3U4BSjJCL/kN68edNuZeGLL774+OOP7U4WPv3009u3b9udLKgLVJdpd4Yn/5Cq8VSjaneywJAm5/79+w8fPrQqBwCA8YLKAQBGDyoHABg9qBwAYPSgcgCA0YPKAQBGDyoHABg9qBwAYPSgcgCA0YPKAQBGDyoHABg9qBwAYPSgcgCA0YPKAQBGDyoHABg9qBwAYPSgcgCA0YPKAQBGDyoHABg9qBwAYPSgcgCA0YPKh+Lk5OSd//Zfc+Z/X/tfDx8+tN0DwC6Byofi3r17/+efH/u//zNfXnz2qdPTU9s9AOwSqHwolMo/e/2xv3/4s2z5p+efQeUAuwkqHwpUDgDZQOVDgcoBIBuofChQOQBkA5UPBSoHgGyg8qFA5QCQDVQ+FKgcALKByodiDCqfTydtTOf2eDIWswPbtmLZfNX9wWyh9k0tc1QfSf8qALYXVD4UY5mVV/40lNIdSOeNZufTQuMaVA4QyyqV1+ZsxQPnFFVPILQxUpVrind5zRuszxPZVp/QbHM+RdkA/Vk3K/cfdPEDvLOMWOVW5qtdLp44o3KA4ZCpXD2+3R48jL9hKg+7uVXlprAqLXYturSYtxfo/UaFFnSd1SqvXor4cwJgx+mucr3V+eniURz3rLymftfButxsu6e1Vmjg1qpYqtw0aD4ITN87f/8AiOiocv2f8KNVPoX2IbaPoqE4yauwbLaoqetUNRRVT7pC0Xut3O1BYw/IGlliWmvXTw9GrfJisLzSctTNULWdVqvQQJ/RPNI6K299TQAQppPKCwKyUzXKI+2PYqPCdClip8qyee2DYrs0Q7FbLy/OdM8TNjI427RWbuqYSssBrJ/WUqFBa0eqflW0rNFeFQCCdF1gKYzY8owWBxx0HfdRXFvBoaxb9uNW09umfLlhC/1mOjUyPCNWeTGCyzK9W+7o7eW7UJa2VmjSdsj9LWLVZstrAoBVCP7sqZ/Etmex5cn1nvPVFQymwJS4D7xbrdrWW7ZGvXFJI4MzUpUXY9gosYNZbTuntVdowT+o9926yzaXGwDQDYHKFdqQ3pOqixoPnXvW2goKt04XCxevQ+O0LG1kYDZL5WbEam+cphpHlxXVDqZTNYCK6m8PejhbK9iT6yzPMniV7LuzbK3xQgAggEzliuI5cx4yU2O5X3xhrp21tkKtRq22W63aVi/BfUUWYSODM5ZZ+Qah353yXQMACatUXs21FPYRq5W1FLrKrPa9CtVuadWqxJnWLQsPZvOyPd2l05qhfPpFjZgzCkyVelECULkYfdfk+ZwF2DbWzco3kcVs5jzvm/r8o3IB5QcuIgeIY3wq976Fq93kE+okoHIAyMY4Z+XlQoliMz2uQOUAkI0xqnwcoHIAyAYqHwpUDgDZQOVDgcoBIBuofChQOQBkA5UPBSoHgGyg8qFA5QCQDVQ+FKgcALKByocClQNANlD5UKByAMgGKh+K4+Nj+w9Sc3HmH/7hhx9+sN0DwC6BygEARg8qBwAYPah8QB7k5fvvv7cdA8COgcqH4vj4+JX9/bcOD7Pl2QsXWCsH2E1Q+VDcu3fvk3v3/vbTT9ny6v4+v2AB2E1Q+VCgcgDIBiofClQOANlA5UOBygEgG6h8KFA5AGQDlQ8FKgeAbKDyoUDlAJANVD4UqBwAsoHKhwKVA0A2UPlQoHIAyAYqH4oxqHw+tf/7uHWmc3s8GYvZgW1bcTBb2GIB+qWWr8vdBgANKh+KsczKjWUrMZbSHUjn0c0u9W0/flA5gAsqH4qRqlxT2HLN5FmfJ/KpPkEwIQ+33+8zAWArQeVDMWKVW5mv9q54lUOo8nD7qBygASofis1SedjNrWI0hVVpsWvRpcW8vUDvNyq0oOsEVL62/ZrZTe3pvDwp0B3AToHKh2LUs/Ka+l0H63Kz7Z7WWqGBW6vG2vaLV1Prrnx14e4AdgpUPhSjVnlhS6/UCrVFtZZahQb6jJXWXdW+u+tvo3IAVD4Y27RWbuqYSstZcP20lgoNVlh3bfu13dA2wA6DyodixCovpsfLMr1b7rSqtrVCA32Gd2Q+0/tr21e4u6FtgB0GlQ/FSFVudl2RFyXWwdW2c1p7hQbOGQVK23q3/XSvtrsb2gbYYVD5UGyWyvVst6lYU9pgRbWD6VSpU1H9gESLtLWCPdmwrFzHVurevnppzvZ82WrjJQPsGKh8KMYyKweALQCVDwUqB4BsoPKhQOUAkA1UPhSoHACygcqHApUDQDZQ+VCgcgDIBiofClQOANlA5UOBygEgG6h8KFA5AGQDlQ8FKgeAbKDyoUDlAJANVD4USuXvHx39/sGDbHnh5z9H5QC7CSofipOTk/9xeJgz/379+sOHD233ALA7/PTT/wcX/pJ8NEj4egAAAABJRU5ErkJggg=="
    }
   },
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Classification\n",
    "### Cross Validation\n",
    "Proses pembentukan model klasifikasi dilakukan dengan bantuan metode evaluasi Validasi silang atau Cross Validation, yang merupakan pendekatan yang populer digunakan untuk evaluasi kinerja untuk pengklasifikasi. Pendekatan umumnya adalah bahwa model dilatih beberapa kali dan untuk setiap kali satu bagian dari keseluruhan diperlakukan sebagai data evaluasi sedangkan sisanya digunakan untuk pelatihan.\n",
    "\n",
    "Metode ini membagi dataset menjadi K bagian yang disebut folds, dengan ukuran yang sama. Setiap fold hanya satu dari N sampel data digunakan untuk tujuan validasi, dan sisanya digunakan sebagai data latih. Proses ini akan diulang sebanyak N kali dan hasil akhirnya adalah rata-rata.\n",
    "\n",
    "![kfold10.png](attachment:kfold10.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Training Model\n",
    "Tahapan berikutnya adalah proses klasifikasi menggunakan gabungan metode Support Vector Machine dan metode Ensemble (AdaBoost) untuk deteksi sarkasme, apakah teks tersebut termasuk sarkasme atau bukan sarkasme."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "================ Model ================ \n",
      "\n",
      "================ Pembagian data Training dan Testing ================ \n",
      "\n",
      "K - Fold Cross Validation\n",
      "fold-berjumlah: 5\n",
      "867\n",
      "\n",
      "================ Fold ke - 1\n",
      "Waktu Eksekusi SMOTE=  0.5160977069990622 detik atau 0.008601628449984371 menit\n",
      "Jumlah Data latih sebelum SMOTE = 480\n",
      "Sarkasme = 73 BukanSarkas = 407\n",
      "Jumlah Data latih setelah SMOTE = 814\n",
      "Sarkasme = 407 BukanSarkas = 407\n",
      "Jumlah Data Uji = 120\n",
      "\n",
      "Metode Adaboost SVM\n",
      "Waktu Eksekusi Training Test=  69.11410175999845 detik atau 1.1519016959999742 menit\n",
      "(TP) TruePositif : 5 \n",
      "(FP) FalsePositif : 8 \n",
      "(TN) TrueNegatif : 78 \n",
      "(FN) FalseNegatif : 29\n",
      "Akurasi Fold ke - 1 = 69.167\n",
      "Presisi Fold ke - 1 = 38.462\n",
      "Recall Fold ke - 1 = 14.706\n",
      "F1-Measure Fold ke - 1 = 21.277\n",
      "867\n",
      "\n",
      "================ Fold ke - 2\n",
      "Waktu Eksekusi SMOTE=  0.15926613400006318 detik atau 0.0026544355666677194 menit\n",
      "Jumlah Data latih sebelum SMOTE = 480\n",
      "Sarkasme = 83 BukanSarkas = 397\n",
      "Jumlah Data latih setelah SMOTE = 794\n",
      "Sarkasme = 397 BukanSarkas = 397\n",
      "Jumlah Data Uji = 120\n",
      "\n",
      "Metode Adaboost SVM\n",
      "Waktu Eksekusi Training Test=  77.2981243739996 detik atau 1.2883020728999932 menit\n",
      "(TP) TruePositif : 5 \n",
      "(FP) FalsePositif : 9 \n",
      "(TN) TrueNegatif : 87 \n",
      "(FN) FalseNegatif : 19\n",
      "Akurasi Fold ke - 2 = 76.667\n",
      "Presisi Fold ke - 2 = 35.714\n",
      "Recall Fold ke - 2 = 20.833\n",
      "F1-Measure Fold ke - 2 = 26.316\n",
      "867\n",
      "\n",
      "================ Fold ke - 3\n",
      "Waktu Eksekusi SMOTE=  1.0123771729995497 detik atau 0.01687295288332583 menit\n",
      "Jumlah Data latih sebelum SMOTE = 480\n",
      "Sarkasme = 97 BukanSarkas = 383\n",
      "Jumlah Data latih setelah SMOTE = 766\n",
      "Sarkasme = 383 BukanSarkas = 383\n",
      "Jumlah Data Uji = 120\n",
      "\n",
      "Metode Adaboost SVM\n",
      "Waktu Eksekusi Training Test=  59.621921706999274 detik atau 0.9936986951166545 menit\n",
      "(TP) TruePositif : 2 \n",
      "(FP) FalsePositif : 4 \n",
      "(TN) TrueNegatif : 106 \n",
      "(FN) FalseNegatif : 8\n",
      "Akurasi Fold ke - 3 = 90.0\n",
      "Presisi Fold ke - 3 = 33.333\n",
      "Recall Fold ke - 3 = 20.0\n",
      "F1-Measure Fold ke - 3 = 25.0\n",
      "867\n",
      "\n",
      "================ Fold ke - 4\n",
      "Waktu Eksekusi SMOTE=  0.13833599999998114 detik atau 0.0023055999999996857 menit\n",
      "Jumlah Data latih sebelum SMOTE = 480\n",
      "Sarkasme = 88 BukanSarkas = 392\n",
      "Jumlah Data latih setelah SMOTE = 784\n",
      "Sarkasme = 392 BukanSarkas = 392\n",
      "Jumlah Data Uji = 120\n",
      "\n",
      "Metode Adaboost SVM\n",
      "Waktu Eksekusi Training Test=  57.88678527999946 detik atau 0.9647797546666577 menit\n",
      "(TP) TruePositif : 0 \n",
      "(FP) FalsePositif : 7 \n",
      "(TN) TrueNegatif : 94 \n",
      "(FN) FalseNegatif : 19\n",
      "Akurasi Fold ke - 4 = 78.333\n",
      "Presisi Fold ke - 4 = 0.0\n",
      "Recall Fold ke - 4 = 0.0\n",
      "F1-Measure Fold ke - 4 = 0\n",
      "867\n",
      "\n",
      "================ Fold ke - 5\n",
      "Waktu Eksekusi SMOTE=  0.142426025999157 detik atau 0.00237376709998595 menit\n",
      "Jumlah Data latih sebelum SMOTE = 480\n",
      "Sarkasme = 87 BukanSarkas = 393\n",
      "Jumlah Data latih setelah SMOTE = 786\n",
      "Sarkasme = 393 BukanSarkas = 393\n",
      "Jumlah Data Uji = 120\n",
      "\n",
      "Metode Adaboost SVM\n",
      "Waktu Eksekusi Training Test=  58.44826538600137 detik atau 0.9741377564333561 menit\n",
      "(TP) TruePositif : 8 \n",
      "(FP) FalsePositif : 5 \n",
      "(TN) TrueNegatif : 95 \n",
      "(FN) FalseNegatif : 12\n",
      "Akurasi Fold ke - 5 = 85.833\n",
      "Presisi Fold ke - 5 = 61.538\n",
      "Recall Fold ke - 5 = 40.0\n",
      "F1-Measure Fold ke - 5 = 48.485\n",
      "rata-rata akurasi= 80.0\n",
      "Waktu Eksekusi Training Test=  322.50406527999985 detik atau 5.375067754666664 menit\n"
     ]
    }
   ],
   "source": [
    "x = newData.iloc[:]\n",
    "y= DataTweet[\"Label\"]\n",
    "\n",
    "print('\\n================ Model ================ ')\n",
    "print('\\n================ Pembagian data Training dan Testing ================ ')\n",
    "#============================ K-fold Start\n",
    "print('\\nK - Fold Cross Validation')\n",
    "k=5\n",
    "kf = KFold(n_splits=k)\n",
    "print(\"fold-berjumlah:\",k)\n",
    "kfold=[]\n",
    "temp_akurasi = []\n",
    "temp_pres = []\n",
    "temp_recall = []\n",
    "temp_f1 = []\n",
    "temp_model=[]\n",
    "\n",
    "it=1\n",
    "TP = 0\n",
    "FP = 0\n",
    "TN = 0\n",
    "FN = 0\n",
    "\n",
    "start = timer()\n",
    "for train_index , test_index in kf.split(x):\n",
    "    startT = timer()\n",
    "    X_train , X_test = x.iloc[train_index,:],x.iloc[test_index,:]\n",
    "    y_train , y_test = y[train_index] , y[test_index]\n",
    "    print(len(X_train.columns))\n",
    "    print('\\n================ Fold ke -',it)\n",
    "    #print('Data Train\\n',X_train)\n",
    "    #print('Data Test\\n',X_test)\n",
    "    startS = timer()\n",
    "    \n",
    "    sm = SMOTE(sampling_strategy=\"minority\",k_neighbors=5)\n",
    "    x_oversample, y_oversample = sm.fit_resample(X_train, y_train)\n",
    "    \n",
    "    endS = timer()\n",
    "    waktu=endS - startS\n",
    "    print(\"Waktu Eksekusi SMOTE= \",waktu,\"detik atau\",waktu/60,\"menit\")\n",
    "    \n",
    "    #setelah resampling dengan SMOTE\n",
    "    jumlah_awal = y_train.shape[0]\n",
    "    cekLabel_awal = Counter(y_train)\n",
    "    jumlah_sm = y_oversample.shape[0]\n",
    "    cekLabel_sm = Counter(y_oversample)\n",
    "    jumlah_tes = y_test.shape[0]\n",
    "    print('Jumlah Data latih sebelum SMOTE =', jumlah_awal)\n",
    "    print('Sarkasme =', cekLabel_awal['Sarkasme'],'BukanSarkas =',cekLabel_awal['BukanSarkas'])\n",
    "    print('Jumlah Data latih setelah SMOTE =', jumlah_sm)\n",
    "    print('Sarkasme =', cekLabel_sm['Sarkasme'],'BukanSarkas =',cekLabel_sm['BukanSarkas'])\n",
    "    print('Jumlah Data Uji =', jumlah_tes)\n",
    "    #================== Metode klasifikasi\n",
    "    baseLearn_svm=SVC(probability=True,kernel='linear')\n",
    "    print('\\nMetode Adaboost SVM')\n",
    "    model_adaboost =AdaBoostClassifier(n_estimators=30, base_estimator=baseLearn_svm,learning_rate=0.5)\n",
    "    model_adaboost.fit(x_oversample,y_oversample)\n",
    "    Prediksi = model_adaboost.predict(X_test)\n",
    "    #add to list model\n",
    "    temp_model.append(model_adaboost)\n",
    "    #=================\n",
    "    ceklah = pd.DataFrame(columns=['Tweet_Split','Label_Split','Label_Prediksi'])\n",
    "    ceklah['newTweet'] = DataTweet['Tweet'].iloc[test_index]\n",
    "    ceklah['Label'] = DataTweet['Label'].iloc[test_index]\n",
    "    ceklah['LabelPrediksi'] = Prediksi\n",
    "    \n",
    "    endT = timer()\n",
    "    waktu=endT - startT\n",
    "    print(\"Waktu Eksekusi Training Test= \",waktu,\"detik atau\",waktu/60,\"menit\")\n",
    "    \n",
    "    jumlahtes = ceklah.shape[0]\n",
    "    positif=\"Sarkasme\"\n",
    "    negatif=\"BukanSarkas\"\n",
    "    for i in range(jumlahtes):\n",
    "        cek=ceklah.iloc[i]\n",
    "        if (cek.Label==positif and cek.LabelPrediksi==positif):\n",
    "            TP+=1\n",
    "        elif(cek.Label==positif and cek.LabelPrediksi==negatif):\n",
    "            FN+=1\n",
    "        elif(cek.Label==negatif and cek.LabelPrediksi==negatif):\n",
    "            TN+=1\n",
    "        elif(cek.Label==negatif and cek.LabelPrediksi==positif):\n",
    "            FP+=1\n",
    "    \n",
    "    print(\"(TP) TruePositif :\",TP,\"\\n(FP) FalsePositif :\",FP,\"\\n(TN) TrueNegatif :\",TN,\"\\n(FN) FalseNegatif :\",FN)\n",
    "    #akurasi\n",
    "    akurasi=(TP+TN)/(TP+FP+TN+FN)\n",
    "    #presisi\n",
    "    cek0 = TP+FP\n",
    "    if(cek0==0):\n",
    "        presisi=0\n",
    "    else:\n",
    "        presisi=TP/(TP+FP)\n",
    "    #recall\n",
    "    cek1 = TP+FN\n",
    "    if(cek1==0):\n",
    "        recal=0\n",
    "    else:\n",
    "        recal=TP/(TP+FN)\n",
    "    \n",
    "    if(presisi==0 and recal==0):\n",
    "        f1=0\n",
    "    else:\n",
    "        f1=(2*presisi*recal)/(presisi+recal)\n",
    "    hasil_akurasi = round(akurasi*100,3)\n",
    "    hasil_pres=round(presisi*100,3)\n",
    "    hasil_recal=round(recal*100,3)\n",
    "    hasil_f1=round(f1*100,3)\n",
    "    \n",
    "    \n",
    "    temp_akurasi.append(hasil_akurasi)\n",
    "    temp_pres.append(hasil_pres)\n",
    "    temp_recall.append(hasil_recal)\n",
    "    temp_f1.append(hasil_f1)\n",
    "    print('Akurasi Fold ke -',it,'=',hasil_akurasi)\n",
    "    print('Presisi Fold ke -',it,'=',hasil_pres)\n",
    "    print('Recall Fold ke -',it,'=',hasil_recal)\n",
    "    print('F1-Measure Fold ke -',it,'=',hasil_f1)\n",
    "    kfold.append([jumlahtes,TP,FP,TN,FN,hasil_akurasi,hasil_pres,hasil_recal,hasil_f1])\n",
    "    it=it+1\n",
    "    TP = 0\n",
    "    FP = 0\n",
    "    TN = 0\n",
    "    FN = 0\n",
    "\n",
    "rata2=0\n",
    "rata2rec=0\n",
    "rata2pre=0\n",
    "rata2f1=0\n",
    "for x in range(len(temp_akurasi)):\n",
    "    rata2=rata2+temp_akurasi[x]\n",
    "print(\"rata-rata akurasi=\",round(rata2/k,3))\n",
    "\n",
    "end = timer()\n",
    "waktue = end - start\n",
    "print(\"Waktu Eksekusi Training Test= \",waktue,\"detik atau\",waktue/60,\"menit\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Saving Model with Pickle\n",
    "Model hasil training sebelumnya dapat disimpan dengan format pickle, model pelatihan dipilih dari data akurasi yang paling tinggi."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MAX akurasi 90.0\n"
     ]
    }
   ],
   "source": [
    "#print(temp_model)\n",
    "maxacc=max(temp_akurasi)\n",
    "print(\"MAX akurasi\",maxacc)\n",
    "\n",
    "maxi = temp_akurasi.index(maxacc)\n",
    "modele=temp_model[maxi]\n",
    "\n",
    "#simpan model yang memiliki akurasi tertinggi pada cross validation\n",
    "\"\"\"\n",
    "import os\n",
    "import os.path\n",
    "pkl_filename = \"pickle_modelF.pkl\"\n",
    "if (os.path.exists(pkl_filename)):\n",
    "    os.remove(pkl_filename)\n",
    "\n",
    "with open(pkl_filename,'wb') as file:\n",
    "    pickle.dump(modele,file)\n",
    "\"\"\"\n",
    "pickle.dump(tfidf_vect.vocabulary_, open(\"pickle_featureF.pkl\", \"wb\"))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Implementasi Model\n",
    "Model yang tersimpan dapat diimplementasikan untuk mendeteksi data text baru. Tahapannya sama mulai dari text-preprocessing sampai dengan Stemming."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "...... Jumlah Data ...... \n",
      "Total: 114\n",
      "   Sarkasme  BukanSarkas\n",
      "0        24           90\n",
      "\n",
      "##-------- Mulai Proses Preprocessing --------##\n",
      "\n",
      "\n",
      "...... Proses Casefolding lowercase, hapus URL...... \n",
      "                                                 Tweet        Label  \\\n",
      "0    sedih bgt anjing gue gabisa scroll tl gabisa l...     Sarkasme   \n",
      "1    By.U ini pakai jaringan @smartfrencare ya min ...  BukanSarkas   \n",
      "2    Kalo jaringan bagus dan aku gaada kerjaan ya b...  BukanSarkas   \n",
      "3    RT @NUgarislucu: Termasuk bangkitnya lagi ITJ ...  BukanSarkas   \n",
      "4    Gue belum dengerin ulang yours💔 tadi kepotong²...  BukanSarkas   \n",
      "..                                                 ...          ...   \n",
      "109  PLISS LAGI SERU SERUNYAAA malah hilanh jaringa...  BukanSarkas   \n",
      "110  @Jaehytun iyaa, ini indosut, tsel, di awa dari...  BukanSarkas   \n",
      "111      Jaringan telkomsel astagfirulloh,bisa2 ny H+😭  BukanSarkas   \n",
      "112  @FirstMediaCares Kasian sekali hanya sibuk men...  BukanSarkas   \n",
      "113                  Kirain jam segini jaringan baguss  BukanSarkas   \n",
      "\n",
      "                                           casefolding  \\\n",
      "0    sedih bgt anjing gue gabisa scroll tl gabisa l...   \n",
      "1    by.u ini pakai jaringan @smartfrencare ya min ...   \n",
      "2    kalo jaringan bagus dan aku gaada kerjaan ya b...   \n",
      "3    rt @nugarislucu: termasuk bangkitnya lagi itj ...   \n",
      "4    gue belum dengerin ulang yours💔 tadi kepotong²...   \n",
      "..                                                 ...   \n",
      "109  pliss lagi seru serunyaaa malah hilanh jaringa...   \n",
      "110  @jaehytun iyaa, ini indosut, tsel, di awa dari...   \n",
      "111      jaringan telkomsel astagfirulloh,bisa2 ny h+😭   \n",
      "112  @firstmediacares kasian sekali hanya sibuk men...   \n",
      "113                  kirain jam segini jaringan baguss   \n",
      "\n",
      "                                             removeURL  \n",
      "0    sedih bgt anjing gue gabisa scroll tl gabisa l...  \n",
      "1                    by.u ini pakai jaringan ya min ?   \n",
      "2    kalo jaringan bagus dan aku gaada kerjaan ya b...  \n",
      "3     : termasuk bangkitnya lagi itj (indonesia tan...  \n",
      "4    gue belum dengerin ulang yours tadi kepotong² ...  \n",
      "..                                                 ...  \n",
      "109  pliss lagi seru serunyaaa malah hilanh jaringa...  \n",
      "110   iyaa, ini indosut, tsel, di awa dari kemaren ...  \n",
      "111       jaringan telkomsel astagfirulloh,bisa2 ny h+  \n",
      "112   kasian sekali hanya sibuk menanyakan no id pe...  \n",
      "113                  kirain jam segini jaringan baguss  \n",
      "\n",
      "[114 rows x 4 columns]\n",
      "\n",
      "...... Tokenisasi ...... \n",
      "                                          Tokenisasi\n",
      "0  [sedih, bgt, anjing, gue, gabisa, scroll, tl, ...\n",
      "1           [by.u, ini, pakai, jaringan, ya, min, ?]\n",
      "\n",
      "...... Proses Casefolding2 hapus angka dan simbol...... \n",
      "                                            Cleaning\n",
      "0  [sedih, bgt, anjing, gue, gabisa, scroll, tl, ...\n",
      "1             [byu, ini, pakai, jaringan, ya, min, ]\n",
      "\n",
      "...... Proses Normalisasi ...... \n",
      "                                         Normalisasi\n",
      "0  [sedih, banget, anjing, saya, tidak, bisa, gul...\n",
      "1       [telkomsel, ini, pakai, jaringan, ya, admin]\n",
      "\n",
      "...... Proses Stopword Removal ...... \n",
      "                                            Stopword\n",
      "0  [sedih, banget, anjing, gulir, timeline, liat,...\n",
      "1            [telkomsel, pakai, jaringan, ya, admin]\n",
      "2  [jaringan, bagus, kerjaan, ya, curi, curi, kad...\n",
      "3                  [bangkitnya, indonesia, jaringan]\n",
      "4  [dengarkan, ulang, milikmu, kepotong, potong, ...\n",
      "5  [iya, ulang, kabar, beranda, jaringan, lumayan...\n",
      "\n",
      "................ Proses Stemming ................ \n",
      "0    [sedih, banget, anjing, gulir, timeline, liat,...\n",
      "1              [telkomsel, pakai, jaringan, ya, admin]\n",
      "2    [jaringan, bagus, kerja, ya, curi, curi, kadan...\n",
      "Name: Stemmed, dtype: object\n"
     ]
    }
   ],
   "source": [
    "#============== Read Data Input\n",
    "filetest=\"#tweet_label_dikit.xlsx\"\n",
    "Prediksites2 = pd.read_excel(filetest)\n",
    "tes=Prediksites2\n",
    "#\n",
    "jumlahData=tes[\"Tweet\"].shape[0] #Jumlah Data\n",
    "jmlSarkas=tes[tes.Label==\"Sarkasme\"].shape[0]\n",
    "jmlNonSarkas=tes[tes.Label==\"BukanSarkas\"].shape[0]\n",
    "d={'Jumlah': [jumlahData], 'Sarkasme': [jmlSarkas],'BukanSarkas':[jmlNonSarkas]}\n",
    "\n",
    "cekJumlah = pd.DataFrame(data=d, columns=['Sarkasme','BukanSarkas'])\n",
    "print('\\n...... Jumlah Data ...... ')\n",
    "print(\"Total:\",jumlahData)\n",
    "print(cekJumlah)\n",
    "#ax = cekJumlah.plot.bar(rot=0)\n",
    "#============== Start Processing Text\n",
    "print(\"\\n##-------- Mulai Proses Preprocessing --------##\\n\")\n",
    "print('\\n...... Proses Casefolding lowercase, hapus URL...... ')\n",
    "tes['casefolding'] = tes['Tweet'].apply(lower)\n",
    "tes['removeURL'] = tes['casefolding'].apply(removeURLemoji)\n",
    "print(tes)\n",
    "\n",
    "#==== Tokenisasi : memisahkan kata dalam kalimat\n",
    "print('\\n...... Tokenisasi ...... ')\n",
    "tes['Tokenisasi'] = tes['removeURL'].apply(tokenize)\n",
    "print(tes[['Tokenisasi']].head(2))\n",
    "\n",
    "print('\\n...... Proses Casefolding2 hapus angka dan simbol...... ')\n",
    "tes['Cleaning'] = tes['Tokenisasi'].apply(hapus_simbolAngka)\n",
    "Prediksites2=pd.DataFrame()\n",
    "Prediksites2['Cleaning']=tes['Cleaning']\n",
    "print(Prediksites2[['Cleaning']].head(2))\n",
    "#============== Normalisasi: kata gaul, singkatan jadi kata baku\n",
    "print('\\n...... Proses Normalisasi ...... ')\n",
    "#normalisasi_dict = normal_term() #import excel\n",
    "tes['Normalisasi'] = tes['Cleaning'].apply(normalisasi)\n",
    "print(tes[['Normalisasi']].head(2))\n",
    "\n",
    "#==== Stopword Removal : hapus kata yang tidak terlalu penting\n",
    "print('\\n...... Proses Stopword Removal ...... ')\n",
    "#list_stopwords = daftarStopword()\n",
    "tes['Stopword'] = tes['Normalisasi'].apply(delstopwordID)\n",
    "print(tes[['Stopword']].head(6))\n",
    "\n",
    "#==== Stemming : mengurangi dimensi fitur kata/term\n",
    "print('\\n................ Proses Stemming ................ ')\n",
    "tes['Stemmed'] = tes['Stopword'].apply(stemming)\n",
    "print(tes['Stemmed'].head(3))\n",
    "tes['newTweet'] = tes['Stemmed'].apply(listokalimat)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load Model TF-IDF dan Klasifikasi\n",
    "Akan ada dua file yang akan digunakan yaitu model dari fitur TF-IDF yang telah tersimpan sesuai dengan model yang dibentuk untuk mentransformasikan teks baru, dan model klasifikasi itu sendiri untuk memprediksi data tersebut."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***Load Model TF-IDF***"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "................ Load Model Fitur TF-IDF ................ \n",
      "\n",
      "================\n",
      "     aba  abad  adik     admin  administrator  aduh   ah      ahli  aja  ajak  \\\n",
      "0    0.0   0.0   0.0  0.000000            0.0   0.0  0.0  0.000000  0.0   0.0   \n",
      "1    0.0   0.0   0.0  0.542116            0.0   0.0  0.0  0.000000  0.0   0.0   \n",
      "2    0.0   0.0   0.0  0.000000            0.0   0.0  0.0  0.000000  0.0   0.0   \n",
      "3    0.0   0.0   0.0  0.000000            0.0   0.0  0.0  0.000000  0.0   0.0   \n",
      "4    0.0   0.0   0.0  0.000000            0.0   0.0  0.0  0.000000  0.0   0.0   \n",
      "..   ...   ...   ...       ...            ...   ...  ...       ...  ...   ...   \n",
      "109  0.0   0.0   0.0  0.000000            0.0   0.0  0.0  0.000000  0.0   0.0   \n",
      "110  0.0   0.0   0.0  0.000000            0.0   0.0  0.0  0.000000  0.0   0.0   \n",
      "111  0.0   0.0   0.0  0.000000            0.0   0.0  0.0  0.000000  0.0   0.0   \n",
      "112  0.0   0.0   0.0  0.000000            0.0   0.0  0.0  0.293638  0.0   0.0   \n",
      "113  0.0   0.0   0.0  0.000000            0.0   0.0  0.0  0.000000  0.0   0.0   \n",
      "\n",
      "     ...    wkwkwk  wow   xd   xl        ya  yakxd  yaudah  youtube  yuk  zoom  \n",
      "0    ...  0.000000  0.0  0.0  0.0  0.000000    0.0     0.0      0.0  0.0   0.0  \n",
      "1    ...  0.000000  0.0  0.0  0.0  0.433360    0.0     0.0      0.0  0.0   0.0  \n",
      "2    ...  0.275516  0.0  0.0  0.0  0.191966    0.0     0.0      0.0  0.0   0.0  \n",
      "3    ...  0.000000  0.0  0.0  0.0  0.000000    0.0     0.0      0.0  0.0   0.0  \n",
      "4    ...  0.000000  0.0  0.0  0.0  0.000000    0.0     0.0      0.0  0.0   0.0  \n",
      "..   ...       ...  ...  ...  ...       ...    ...     ...      ...  ...   ...  \n",
      "109  ...  0.000000  0.0  0.0  0.0  0.000000    0.0     0.0      0.0  0.0   0.0  \n",
      "110  ...  0.000000  0.0  0.0  0.0  0.000000    0.0     0.0      0.0  0.0   0.0  \n",
      "111  ...  0.000000  0.0  0.0  0.0  0.000000    0.0     0.0      0.0  0.0   0.0  \n",
      "112  ...  0.000000  0.0  0.0  0.0  0.000000    0.0     0.0      0.0  0.0   0.0  \n",
      "113  ...  0.000000  0.0  0.0  0.0  0.000000    0.0     0.0      0.0  0.0   0.0  \n",
      "\n",
      "[114 rows x 867 columns]\n"
     ]
    }
   ],
   "source": [
    "print('\\n................ Load Model Fitur TF-IDF ................ ')\n",
    "savedtfidf = pickle.load(open(\"pickle_featureF.pkl\", 'rb'))\n",
    "vectorizer2 = TfidfVectorizer(vocabulary=savedtfidf)\n",
    "vect_docs2 = vectorizer2.fit_transform(tes['newTweet'])\n",
    "features_names2 = vectorizer2.get_feature_names_out()\n",
    "\n",
    "dense2 = vect_docs2.todense()\n",
    "alist2 = dense2.tolist()\n",
    "print('\\n================')\n",
    "newData2 = pd.DataFrame(alist2,columns=features_names2)\n",
    "print(newData2)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***Load Model Klasifikasi***"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Tweet</th>\n",
       "      <th>Label</th>\n",
       "      <th>Prediksi</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>sedih bgt anjing gue gabisa scroll tl gabisa l...</td>\n",
       "      <td>Sarkasme</td>\n",
       "      <td>Sarkasme</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>By.U ini pakai jaringan @smartfrencare ya min ...</td>\n",
       "      <td>BukanSarkas</td>\n",
       "      <td>BukanSarkas</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Kalo jaringan bagus dan aku gaada kerjaan ya b...</td>\n",
       "      <td>BukanSarkas</td>\n",
       "      <td>BukanSarkas</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>RT @NUgarislucu: Termasuk bangkitnya lagi ITJ ...</td>\n",
       "      <td>BukanSarkas</td>\n",
       "      <td>BukanSarkas</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Gue belum dengerin ulang yours💔 tadi kepotong²...</td>\n",
       "      <td>BukanSarkas</td>\n",
       "      <td>BukanSarkas</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>109</th>\n",
       "      <td>PLISS LAGI SERU SERUNYAAA malah hilanh jaringa...</td>\n",
       "      <td>BukanSarkas</td>\n",
       "      <td>BukanSarkas</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>110</th>\n",
       "      <td>@Jaehytun iyaa, ini indosut, tsel, di awa dari...</td>\n",
       "      <td>BukanSarkas</td>\n",
       "      <td>BukanSarkas</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>111</th>\n",
       "      <td>Jaringan telkomsel astagfirulloh,bisa2 ny H+😭</td>\n",
       "      <td>BukanSarkas</td>\n",
       "      <td>BukanSarkas</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>112</th>\n",
       "      <td>@FirstMediaCares Kasian sekali hanya sibuk men...</td>\n",
       "      <td>BukanSarkas</td>\n",
       "      <td>BukanSarkas</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>113</th>\n",
       "      <td>Kirain jam segini jaringan baguss</td>\n",
       "      <td>BukanSarkas</td>\n",
       "      <td>BukanSarkas</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>114 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                 Tweet        Label  \\\n",
       "0    sedih bgt anjing gue gabisa scroll tl gabisa l...     Sarkasme   \n",
       "1    By.U ini pakai jaringan @smartfrencare ya min ...  BukanSarkas   \n",
       "2    Kalo jaringan bagus dan aku gaada kerjaan ya b...  BukanSarkas   \n",
       "3    RT @NUgarislucu: Termasuk bangkitnya lagi ITJ ...  BukanSarkas   \n",
       "4    Gue belum dengerin ulang yours💔 tadi kepotong²...  BukanSarkas   \n",
       "..                                                 ...          ...   \n",
       "109  PLISS LAGI SERU SERUNYAAA malah hilanh jaringa...  BukanSarkas   \n",
       "110  @Jaehytun iyaa, ini indosut, tsel, di awa dari...  BukanSarkas   \n",
       "111      Jaringan telkomsel astagfirulloh,bisa2 ny H+😭  BukanSarkas   \n",
       "112  @FirstMediaCares Kasian sekali hanya sibuk men...  BukanSarkas   \n",
       "113                  Kirain jam segini jaringan baguss  BukanSarkas   \n",
       "\n",
       "        Prediksi  \n",
       "0       Sarkasme  \n",
       "1    BukanSarkas  \n",
       "2    BukanSarkas  \n",
       "3    BukanSarkas  \n",
       "4    BukanSarkas  \n",
       "..           ...  \n",
       "109  BukanSarkas  \n",
       "110  BukanSarkas  \n",
       "111  BukanSarkas  \n",
       "112  BukanSarkas  \n",
       "113  BukanSarkas  \n",
       "\n",
       "[114 rows x 3 columns]"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Load from file\n",
    "\n",
    "import pickle\n",
    "\n",
    "with open(\"pickle_modelF.pkl\", 'rb') as file:\n",
    "    pickle_model = pickle.load(file)\n",
    "hasil=pickle_model.predict(newData2)\n",
    "DFpredict = pd.DataFrame(hasil,columns=[\"Prediksi\"])\n",
    "#print(DFpredict)\n",
    "gabungkan = pd.concat([tes[['Tweet','Label']], DFpredict], axis=1)\n",
    "gabungkan\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}