dogs_vs_cats_image_classification.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/MinahilSadiq1/Image_Classification_Raw_Pixels_vs._High-Level_Feature_Extractions/blob/main/dogs_vs_cats_image_classification.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "z5n4aJYpGmkb"
      },
      "source": [
        "**Minahil Sadiq**\n",
        "\n",
        "**SP20-BCS-023**\n",
        "\n",
        "**SECTION: B**\n",
        "\n",
        "# ***ASSIGNMENT 2***"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "#### the dataset used is taken from kaggle 'dogs vs cats', https://www.kaggle.com/c/dogs-vs-cats"
      ],
      "metadata": {
        "id": "lsRa5I_YBMHM"
      }
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QytJ_cRrLjZ0"
      },
      "source": [
        "DOWNLOADING AND EXTRACTING DATASET"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "RrF5dTbkKgj3",
        "outputId": "2396eab1-948d-4d27-c7ae-dd49021ceb4b"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Drive already mounted at /content/drive; to attempt to forcibly remount, call drive.mount(\"/content/drive\", force_remount=True).\n"
          ]
        }
      ],
      "source": [
        "from google.colab import drive\n",
        "drive.mount('/content/drive')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lvdpZXMELvjG"
      },
      "source": [
        "IMPORTING LIBRARIES"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "0Lr9gy_bLsqx"
      },
      "outputs": [],
      "source": [
        "import os\n",
        "import cv2\n",
        "import numpy as np\n",
        "import pandas as pd\n",
        "import zipfile\n",
        "from tqdm import tqdm\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.model_selection import GridSearchCV\n",
        "from sklearn.model_selection import RandomizedSearchCV\n",
        "from sklearn.linear_model import LogisticRegression\n",
        "from sklearn.svm import SVC\n",
        "from sklearn.ensemble import RandomForestClassifier\n",
        "from sklearn.metrics import accuracy_score\n",
        "from sklearn.preprocessing import StandardScaler\n",
        "from sklearn.decomposition import PCA\n",
        "from skimage.feature import hog\n",
        "from scipy.stats import loguniform"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "XaDyG46fLL9s"
      },
      "outputs": [],
      "source": [
        "zip_ref = zipfile.ZipFile('/content/drive/MyDrive/train.zip', 'r') #Opens the zip file in read mode\n",
        "zip_ref.extractall('/tmp') #Extracts the files into the /tmp folder\n",
        "zip_ref.close()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "odrX9S2lOqHU",
        "outputId": "ada18fd5-c4d0-462d-f3c6-f203e327e728"
      },
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "25000"
            ]
          },
          "metadata": {},
          "execution_count": 42
        }
      ],
      "source": [
        "len(os.listdir('/tmp/train/'))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tMT0z6ieImMB"
      },
      "source": [
        "1: USING RAW PIXELS AS FEATURES"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "RZomBqZ8Gduq",
        "outputId": "8c5bc769-8bde-4e94-dd12-d6487bbcf140"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 25000/25000 [00:31<00:00, 794.40it/s]\n"
          ]
        }
      ],
      "source": [
        "# Load the dataset\n",
        "size = 64\n",
        "X, y = [], []\n",
        "for file in tqdm(os.listdir('/tmp/train/')):\n",
        "    img = cv2.imread(os.path.join('/tmp/train/', file), cv2.IMREAD_GRAYSCALE)\n",
        "    if img is None:\n",
        "        continue # skip this if it can't be read\n",
        "    img = cv2.resize(img, (size, size))\n",
        "    X.append(img.flatten())\n",
        "    label = 1 if 'dog' in file else 0\n",
        "    y.append(label)\n",
        "\n",
        "\n",
        "# Convert to numpy arrays\n",
        "X = np.array(X)\n",
        "y = np.array(y)\n",
        "\n",
        "# Split into train and test sets\n",
        "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "QmuwdOGsT6m1"
      },
      "outputs": [],
      "source": [
        "# Logistic Regression\n",
        "lr = LogisticRegression(max_iter = 1000, random_state=1000)\n",
        "lr.fit(X_train, y_train)\n",
        "lr_pred = lr.predict(X_test)\n",
        "lr_acc = accuracy_score(y_test, lr_pred)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "GOsdB3SnUFgj"
      },
      "outputs": [],
      "source": [
        "# SVM\n",
        "scaler = StandardScaler()\n",
        "X_train_scaled = scaler.fit_transform(X_train)\n",
        "X_test_scaled = scaler.transform(X_test)\n",
        "\n",
        "svm = SVC(kernel='linear', max_iter=1000, random_state=1000)\n",
        "svm.fit(X_train_scaled, y_train)\n",
        "svm_pred = svm.predict(X_test_scaled)\n",
        "svm_acc = accuracy_score(y_test, svm_pred)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "UN22yw6dUKLf"
      },
      "outputs": [],
      "source": [
        "# Random Forest\n",
        "rf = RandomForestClassifier(n_estimators=100, random_state=1000)\n",
        "rf.fit(X_train, y_train)\n",
        "rf_pred = rf.predict(X_test)\n",
        "rf_acc = accuracy_score(y_test, rf_pred)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "DHsGZEUZUNfx",
        "outputId": "fc84f54e-9c14-428a-9c04-f63aea9344df"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Logistic Regression Accuracy: 52.54%\n",
            "SVM Accuracy: 51.42%\n",
            "Random Forest Accuracy: 63.54%\n"
          ]
        }
      ],
      "source": [
        "print(\"Logistic Regression Accuracy: {:.2f}%\".format(lr_acc * 100))\n",
        "print(\"SVM Accuracy: {:.2f}%\".format(svm_acc * 100))\n",
        "print(\"Random Forest Accuracy: {:.2f}%\".format(rf_acc * 100))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "p53HzP5dME5S"
      },
      "source": [
        "2: EXTRACT HIGH-LEVEL FEATURES AT LEAST 2 HIGH-LEVEL FEATURE TECHNIQUES, eg hog, orb etc"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "DequAxiaMhFQ",
        "outputId": "23ad9f1a-486b-4b90-889e-b4a5d12d9d76"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 25000/25000 [01:41<00:00, 246.52it/s]\n"
          ]
        }
      ],
      "source": [
        "# HOG ----------------------------\n",
        "size = 64\n",
        "def extract_hog_features(image):\n",
        "    features = hog(image, orientations=9, pixels_per_cell=(8, 8), cells_per_block=(2, 2), transform_sqrt=True, feature_vector=True)\n",
        "    return features\n",
        "\n",
        "X_hog, y_hog = [], []\n",
        "for file in tqdm(os.listdir('/tmp/train/')):\n",
        "    img = cv2.imread(os.path.join('/tmp/train/', file), cv2.IMREAD_GRAYSCALE)\n",
        "    if img is None:\n",
        "        continue # skip this file if it can't be read\n",
        "    img = cv2.resize(img, (size, size))\n",
        "    features = extract_hog_features(img)\n",
        "    X_hog.append(features)\n",
        "    label = 1 if 'dog' in file else 0\n",
        "    y_hog.append(label)\n",
        "\n",
        "X_hog = np.array(X_hog)\n",
        "y_hog = np.array(y_hog)\n",
        "\n",
        "#SPLITTING\n",
        "X_hog_train, X_hog_test, y_hog_train, y_hog_test = train_test_split(X_hog, y_hog, test_size=0.2, random_state=42)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "q4MKa3c047yz"
      },
      "source": [
        "have done the below code block to solve issue of 'SURF' but it did not get resolved."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "9WxispwzU6xB",
        "outputId": "bc2a3dd3-36d6-4b78-b093-a9c7edf374fc"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "100%|██████████| 25000/25000 [02:39<00:00, 156.97it/s]\n"
          ]
        }
      ],
      "source": [
        "# ORB --------------------------\n",
        "def extract_orb_features(image):\n",
        "    orb = cv2.ORB_create()\n",
        "    keypoints, descriptors = orb.detectAndCompute(image, None)\n",
        "    if descriptors is None:\n",
        "        descriptors = np.zeros((1, 32))\n",
        "    return descriptors.flatten()\n",
        "\n",
        "\n",
        "# Histogram of ORB features\n",
        "def orb_histogram(image):\n",
        "    features = extract_orb_features(image)\n",
        "    hist, _ = np.histogram(features, bins=16, range=(0, 255))\n",
        "    return hist\n",
        "\n",
        "X_orb, y_orb = [], []\n",
        "for file in tqdm(os.listdir('/tmp/train/')):\n",
        "    img = cv2.imread(os.path.join('/tmp/train/', file), cv2.IMREAD_GRAYSCALE)\n",
        "    img = cv2.resize(img, (256, 256))\n",
        "    hist = orb_histogram(img)\n",
        "    X_orb.append(hist)\n",
        "    label = 1 if 'dog' in file else 0\n",
        "    y_orb.append(label)\n",
        "\n",
        "X_orb = np.array(X_orb)\n",
        "y_orb = np.array(y_orb)\n",
        "\n",
        "# Train-test split\n",
        "X_orb_train, X_orb_test, y_orb_train, y_orb_test = train_test_split(X_orb, y_orb, test_size=0.2, random_state=42)\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "n9UseUdBVBt5"
      },
      "outputs": [],
      "source": [
        "# after hog, orb .. now training three models SVM, LOGISTIC REGRESSION, RANDOM FOREST\n",
        "\n",
        "# Logistic Regression\n",
        "#HOG\n",
        "lr_hog = LogisticRegression(max_iter=1000, random_state=42)\n",
        "lr_hog.fit(X_hog_train, y_hog_train)\n",
        "lr_hog_pred = lr_hog.predict(X_hog_test)\n",
        "lr_hog_acc = accuracy_score(y_hog_test, lr_hog_pred)\n",
        "\n",
        "#ORB\n",
        "lr_orb = LogisticRegression(max_iter=1000, random_state=42)\n",
        "lr_orb.fit(X_orb_train, y_orb_train)\n",
        "lr_orb_pred = lr_orb.predict(X_orb_test)\n",
        "lr_orb_acc = accuracy_score(y_orb_test, lr_orb_pred)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "440TIryuVIxH"
      },
      "outputs": [],
      "source": [
        "# SVM\n",
        "#HOG\n",
        "scaler = StandardScaler()\n",
        "X_hog_train_scaled = scaler.fit_transform(X_hog_train)\n",
        "X_hog_test_scaled = scaler.transform(X_hog_test)\n",
        "\n",
        "svm_hog = SVC(kernel='linear', max_iter=1000, random_state=42)\n",
        "svm_hog.fit(X_hog_train_scaled, y_hog_train)\n",
        "svm_hog_pred = svm_hog.predict(X_hog_test_scaled)\n",
        "svm_hog_acc = accuracy_score(y_hog_test, svm_hog_pred)\n",
        "\n",
        "#ORB\n",
        "scaler = StandardScaler()\n",
        "X_orb_train_scaled = scaler.fit_transform(X_orb_train)\n",
        "X_orb_test_scaled = scaler.transform(X_orb_test)\n",
        "\n",
        "svm_orb = SVC(kernel='linear',max_iter=1000, random_state=42)\n",
        "svm_orb.fit(X_orb_train_scaled, y_orb_train)\n",
        "svm_orb_pred = svm_orb.predict(X_orb_test_scaled)\n",
        "svm_orb_acc = accuracy_score(y_orb_test, svm_orb_pred)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "uPphElRKVOui"
      },
      "outputs": [],
      "source": [
        "# Random Forest\n",
        "#HOG\n",
        "rf_hog = RandomForestClassifier(n_estimators=100, random_state=42)\n",
        "rf_hog.fit(X_hog_train, y_hog_train)\n",
        "rf_hog_pred = rf_hog.predict(X_hog_test)\n",
        "rf_hog_acc = accuracy_score(y_hog_test, rf_hog_pred)\n",
        "\n",
        "#ORB\n",
        "rf_orb = RandomForestClassifier(n_estimators=100, random_state=42)\n",
        "rf_orb.fit(X_orb_train, y_orb_train)\n",
        "rf_orb_pred = rf_orb.predict(X_orb_test)\n",
        "rf_orb_acc = accuracy_score(y_orb_test, rf_orb_pred)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "JL-I9VVmVSa3",
        "outputId": "89edbb01-b28e-421f-e855-c1820d95aa36"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Logistic Regression (HOG) Accuracy: 72.30%\n",
            "Logistic Regression (ORB) Accuracy: 63.62%\n",
            "\n",
            "SVM (HOG) Accuracy: 55.26%\n",
            "SVM (ORB) Accuracy: 44.84%\n",
            "\n",
            "RANDOM FOREST (HOG) Accuracy: 69.78%\n",
            "RANDOM FOREST (ORB) Accuracy: 63.74%\n",
            "\n"
          ]
        }
      ],
      "source": [
        "#accuracy score of logistic regression\n",
        "print(\"Logistic Regression (HOG) Accuracy: {:.2f}%\".format(lr_hog_acc * 100))\n",
        "print(\"Logistic Regression (ORB) Accuracy: {:.2f}%\".format(lr_orb_acc * 100))\n",
        "print()\n",
        "#accuracy score of svm\n",
        "print(\"SVM (HOG) Accuracy: {:.2f}%\".format(svm_hog_acc * 100))\n",
        "print(\"SVM (ORB) Accuracy: {:.2f}%\".format(svm_orb_acc * 100))\n",
        "print()\n",
        "#accuracy score of randomforest\n",
        "print(\"RANDOM FOREST (HOG) Accuracy: {:.2f}%\".format(rf_hog_acc * 100))\n",
        "print(\"RANDOM FOREST (ORB) Accuracy: {:.2f}%\".format(rf_orb_acc * 100))\n",
        "print()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dpmvMUKGQm90"
      },
      "source": [
        "3: OPTIMIZE HYPERPARAMETERS"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "LLgS0q_yQuJ1"
      },
      "outputs": [],
      "source": [
        "# Logistic Regression Hyperparameter Tuning\n",
        "#HOG\n",
        "lr_params = {'C': [0.01, 0.1, 1, 10]}\n",
        "lr_hog_grid = GridSearchCV(lr_hog, lr_params, cv=5)\n",
        "lr_hog_grid.fit(X_hog_train, y_hog_train)\n",
        "lr_hog_best = lr_hog_grid.best_estimator_\n",
        "lr_hog_best_pred = lr_hog_best.predict(X_hog_test)\n",
        "lr_hog_best_acc = accuracy_score(y_hog_test, lr_hog_best_pred)\n",
        "\n",
        "#ORB\n",
        "lr_orb_grid = GridSearchCV(lr_orb, lr_params, cv=5)\n",
        "lr_orb_grid.fit(X_orb_train, y_orb_train)\n",
        "lr_orb_best = lr_orb_grid.best_estimator_\n",
        "lr_orb_best_pred = lr_orb_best.predict(X_orb_test)\n",
        "lr_orb_best_acc = accuracy_score(y_orb_test, lr_orb_best_pred)\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "mU3wjL1CDMSL"
      },
      "outputs": [],
      "source": [
        "# SVM Hyperparameter Tuning\n",
        "# define the hyperparameter search space\n",
        "svm_params = {'C': [0.01, 0.1, 1, 10], 'kernel': ['rbf', 'linear',]}\n",
        "#HOG\n",
        "svm_hog_grid = RandomizedSearchCV(\n",
        "    svm_hog,\n",
        "    param_distributions=svm_params,\n",
        "    n_iter=20,\n",
        "    cv=5,\n",
        "    n_jobs=-1,\n",
        "    random_state=42\n",
        ")\n",
        "svm_hog_grid.fit(X_hog_train_scaled, y_hog_train)\n",
        "svm_hog_best = svm_hog_grid.best_estimator_\n",
        "svm_hog_best_pred = svm_hog_best.predict(X_hog_test_scaled)\n",
        "svm_hog_best_acc = accuracy_score(y_hog_test, svm_hog_best_pred)\n",
        "\n",
        "#ORB\n",
        "svm_orb_grid = RandomizedSearchCV(svm_orb, param_distributions=svm_params,\n",
        "    n_iter=1000,\n",
        "    cv=5,\n",
        "    n_jobs=-1,\n",
        "    random_state=42)\n",
        "svm_orb_grid.fit(X_orb_train_scaled, y_orb_train)\n",
        "svm_orb_best = svm_orb_grid.best_estimator_\n",
        "svm_orb_best_pred = svm_orb_best.predict(X_orb_test_scaled)\n",
        "svm_orb_best_acc = accuracy_score(y_orb_test, svm_orb_best_pred)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Eh2UlXNBDScC"
      },
      "outputs": [],
      "source": [
        "# Random Forest Hyperparameter Tuning\n",
        "# Define the hyperparameter search space\n",
        "rf_params = {'n_estimators': [10, 50], 'max_depth': [None, 100, 200, 300], 'min_samples_split': [300, 600, 1000]}\n",
        "# Create the random forest classifier\n",
        "rf_hogg = RandomForestClassifier(random_state=42)\n",
        "# Create the randomized search object\n",
        "rf_hog_random = RandomizedSearchCV(\n",
        "    estimator=rf_hogg,\n",
        "    param_distributions=rf_params,\n",
        "    n_iter=20,\n",
        "    cv=5,\n",
        "    n_jobs=-1,\n",
        "    random_state=42\n",
        ")\n",
        "# Fit the randomized search object to the training data\n",
        "rf_hog_random.fit(X_hog_train, y_hog_train)\n",
        "\n",
        "rf_hog_best = rf_hog_random.best_estimator_\n",
        "rf_hog_best_pred = rf_hog_best.predict(X_hog_test)\n",
        "rf_hog_best_acc = accuracy_score(y_hog_test, rf_hog_best_pred)\n",
        "\n",
        "\n",
        "# Create the random forest classifier\n",
        "rf_orbb = RandomForestClassifier(random_state=42)\n",
        "# Create the randomized search object\n",
        "rf_orb_random = RandomizedSearchCV(\n",
        "    estimator=rf_orbb,\n",
        "    param_distributions=rf_params,\n",
        "    n_iter=20,\n",
        "    cv=5,\n",
        "    n_jobs=-1,\n",
        "    random_state=42\n",
        ")\n",
        "\n",
        "rf_orb_random.fit(X_orb_train, y_orb_train)\n",
        "rf_orb_best = rf_orb_random.best_estimator_\n",
        "rf_orb_best_pred = rf_orb_best.predict(X_orb_test)\n",
        "rf_orb_best_acc = accuracy_score(y_orb_test, rf_orb_best_pred)\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "TekzbYWADXn6",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "d6e98a6a-2a48-4c4c-c690-6dea8a76b554"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Logistic Regression Results:\n",
            "HOG Features: 72.42 with {'C': 0.1, 'class_weight': None, 'dual': False, 'fit_intercept': True, 'intercept_scaling': 1, 'l1_ratio': None, 'max_iter': 1000, 'multi_class': 'auto', 'n_jobs': None, 'penalty': 'l2', 'random_state': 42, 'solver': 'lbfgs', 'tol': 0.0001, 'verbose': 0, 'warm_start': False}\n",
            "ORB Features: 62.38 with {'C': 0.01, 'class_weight': None, 'dual': False, 'fit_intercept': True, 'intercept_scaling': 1, 'l1_ratio': None, 'max_iter': 1000, 'multi_class': 'auto', 'n_jobs': None, 'penalty': 'l2', 'random_state': 42, 'solver': 'lbfgs', 'tol': 0.0001, 'verbose': 0, 'warm_start': False}\n",
            "\n",
            "SVM Results:\n",
            "HOG Features: 63.46000000000001 with {'C': 1, 'break_ties': False, 'cache_size': 200, 'class_weight': None, 'coef0': 0.0, 'decision_function_shape': 'ovr', 'degree': 3, 'gamma': 'scale', 'kernel': 'rbf', 'max_iter': 1000, 'probability': False, 'random_state': 42, 'shrinking': True, 'tol': 0.001, 'verbose': False}\n",
            "ORB Features: 56.279999999999994 with {'C': 0.1, 'break_ties': False, 'cache_size': 200, 'class_weight': None, 'coef0': 0.0, 'decision_function_shape': 'ovr', 'degree': 3, 'gamma': 'scale', 'kernel': 'rbf', 'max_iter': 1000, 'probability': False, 'random_state': 42, 'shrinking': True, 'tol': 0.001, 'verbose': False}\n",
            "\n",
            "Random Forest Results:\n",
            "HOG Features: 69.5 with {'bootstrap': True, 'ccp_alpha': 0.0, 'class_weight': None, 'criterion': 'gini', 'max_depth': 200, 'max_features': 'sqrt', 'max_leaf_nodes': None, 'max_samples': None, 'min_impurity_decrease': 0.0, 'min_samples_leaf': 1, 'min_samples_split': 300, 'min_weight_fraction_leaf': 0.0, 'n_estimators': 50, 'n_jobs': None, 'oob_score': False, 'random_state': 42, 'verbose': 0, 'warm_start': False}\n",
            "ORB Features: 63.68000000000001 with {'bootstrap': True, 'ccp_alpha': 0.0, 'class_weight': None, 'criterion': 'gini', 'max_depth': 200, 'max_features': 'sqrt', 'max_leaf_nodes': None, 'max_samples': None, 'min_impurity_decrease': 0.0, 'min_samples_leaf': 1, 'min_samples_split': 300, 'min_weight_fraction_leaf': 0.0, 'n_estimators': 50, 'n_jobs': None, 'oob_score': False, 'random_state': 42, 'verbose': 0, 'warm_start': False}\n"
          ]
        }
      ],
      "source": [
        "# Print results\n",
        "print(\"Logistic Regression Results:\")\n",
        "print(f\"HOG Features: {lr_hog_best_acc * 100} with {lr_hog_best.get_params()}\")\n",
        "print(f\"ORB Features: {lr_orb_best_acc * 100} with {lr_orb_best.get_params()}\\n\")\n",
        "\n",
        "print(\"SVM Results:\")\n",
        "print(f\"HOG Features: {svm_hog_best_acc * 100} with {svm_hog_best.get_params()}\")\n",
        "print(f\"ORB Features: {svm_orb_best_acc * 100} with {svm_orb_best.get_params()}\\n\")\n",
        "\n",
        "print(\"Random Forest Results:\")\n",
        "print(f\"HOG Features: {rf_hog_best_acc * 100} with {rf_hog_best.get_params()}\")\n",
        "print(f\"ORB Features: {rf_orb_best_acc * 100} with {rf_orb_best.get_params()}\")"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# **Methods and Results**\n",
        "First import all the required Libraries, then load data from the drive.\n",
        "\n",
        "**Part 1:** raw pixels used as features, we chose 3 classifiers *Logistic Regression*, *SVM*, *RandomForest*.after that applied a *for loop* in which we covert the labels of images into the binary form(0,1). Dog images labeled as 1 and cat images labeled as 0. we divide the data into training and testing with the ratio of 80:20.\n",
        "\n",
        "For first model *Logistic Regression*, we set maximum iterations to 1000 as our data is of 25000 images. we trained the model using .fit method. and then we test it on unseen data. it provides us the accuracy of **52.54%**.\n",
        "\n",
        "For *SVM*, we set max_iter to 1000, and used *linear kernel* trained and test the model and it gives the accuracy of **51.42%**.\n",
        "\n",
        "For *RandomForest* we set n_estimators to 100 as decreasing no. of it will make the training faster. after training the model, we test it and it gives the accuracy of **63.54%**.\n",
        "\n",
        "**Part 2:** extracting high level features using techniques such as hog, orb.\n",
        "\n",
        "*Hog*: function 'extract_hog_features'takes input image and create Histogram of Oriented Gradients (HOG) features using 'hog' function. the parameters used in this are orientations, pixels_per_cell etc.. then we make two arrays x_hog and y_hog which store the hog features of image and the resultanat labels of it respectively. resizing image and then function created above will be used on it. then it will label the data in binary form which will make it able to work for classifiers.\n",
        "\n",
        "*Orb*: just like Hog but it will definitely use its own properties and parameters but working is same like Hog except where we use the hog now will use Orb.\n",
        "\n",
        "For *Logistic Regression* it gives us the accuracy of **72.30%** with **HOG** feature technique.\n",
        "\n",
        "For *Logistic Regression* it give us accuracy of **63.62%** with **ORB** feature technique.\n",
        "\n",
        "For *SVM* it give accuracy **55.26%** using **HOG** fetaure tecnique.\n",
        "\n",
        "For *SVM* it give accuracy **44.84%** using **ORB**.\n",
        "\n",
        "For *RandomForest* accuracy= **69.78%** using **HOB**.\n",
        "\n",
        "For *RadomForest* accuracy= **63.74%** using **ORB**.\n",
        "\n",
        "**Part 3:** Optimizing hyperparameters\n",
        "\n",
        "*logistic regression model* to classify images of dogs and cats. It is trying different values of a hyperparameter called 'C' to find the best value for the model. The 'GridSearchCV' function is used to search through a grid of possible 'C' values and find the one that gives the highest accuracy. The best model is then used to predict the labels of a test set of images. It gives **72.42%** accuracy with the best parameter among all **c=0.1** using \"HOG\".\n",
        "\n",
        "*logistic regression model* using \"ORB\" provide accuracy **62.38%** with **c=0.01**.\n",
        "\n",
        "For *SVM* two kernels used first linear nad poly in which it returns linear for both HOG and orb.\n",
        "\n",
        "*SVM* using \"HOG\" accuracy=**59.0%** with **c=0.01, kernel= linear**.\n",
        "\n",
        "*SVM* using \"orb\" accuracy=**56.26%** with **c=10, kernel= linear**.\n",
        "\n",
        "Then we use other two kernels linear and rbf then it returns rbf as best for both.\n",
        "\n",
        "*SVM* using \"HOG\" accuracy=**63.4%** with **c=1, kernel= rbf**.\n",
        "\n",
        "*SVM* using \"orb\" accuracy=**56.26%** with **c=0.1, kernel= rbf**.\n",
        "\n",
        "*RandomForest* \"HOG\" accuracy=**69.5%** with **max depth = 200, max feature= sqrt, min samples split= 300**.\n",
        "\n",
        "*RandomForest* \"ORB\" accuracy=**63.68%** with **max depth = 200, max feature= sqrt, min samples split= 300**.\n"
      ],
      "metadata": {
        "id": "_R-ngeoQJ4yv"
      }
    }
  ],
  "metadata": {
    "colab": {
      "provenance": [],
      "authorship_tag": "ABX9TyNtMGHRBEpmKNC9CPflolQG",
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}