exercise12_MulticlassClassification_CategoricalCrossentropy.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "3aef95bd-19b9-406e-a860-b1720c26d307",
   "metadata": {},
   "source": [
    "Sascha Spors,\n",
    "Professorship Signal Theory and Digital Signal Processing,\n",
    "Institute of Communications Engineering (INT),\n",
    "Faculty of Computer Science and Electrical Engineering (IEF),\n",
    "University of Rostock,\n",
    "Germany\n",
    "\n",
    "# Data Driven Audio Signal Processing - A Tutorial with Computational Examples\n",
    "\n",
    "Winter Semester 2023/24 (Master Course #24512)\n",
    "\n",
    "- lecture: https://github.com/spatialaudio/data-driven-audio-signal-processing-lecture\n",
    "- tutorial: https://github.com/spatialaudio/data-driven-audio-signal-processing-exercise\n",
    "\n",
    "Feel free to contact lecturer frank.schultz@uni-rostock.de"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "46f6f98a-12a4-4bcd-975e-18a7d404cb9c",
   "metadata": {},
   "source": [
    "# Multiclass Classification\n",
    "- One Hot encoding\n",
    "- Data set splitting into train, test\n",
    "- **Softmax** activation function at output layer\n",
    "- categorical cross-entropy loss\n",
    "- we use convenient stuff from scikit-learn"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a66d640a-deef-43b4-9572-491510924933",
   "metadata": {},
   "source": [
    "## Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fcdf6b91-8051-4962-b5ee-c67cd6916135",
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "from sklearn.datasets import make_classification\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.preprocessing import OneHotEncoder, LabelBinarizer\n",
    "import tensorflow as tf\n",
    "from tensorflow import keras\n",
    "\n",
    "print(\n",
    "    \"TF version\",\n",
    "    tf.__version__,\n",
    ")\n",
    "\n",
    "tf.keras.backend.set_floatx(\"float64\")  # we could use double precision\n",
    "\n",
    "verbose = 1  # plot training status"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8902728d-14a9-4f64-8f49-364953c88f08",
   "metadata": {},
   "source": [
    "## Data Synthesis, One Hot Encoding, Train/TestSplitting"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3696b9b2-a2e5-467c-a7b4-3c85fad5cd78",
   "metadata": {},
   "outputs": [],
   "source": [
    "nlabels = 3  # number of classes\n",
    "labels = np.arange(nlabels)  # we encode as integers\n",
    "\n",
    "m = int(5 / 4 * 80000)  # data examples\n",
    "nx = 2 * nlabels  # number of features, we set it to 6 here\n",
    "\n",
    "train_size = 4 / 5  # 80% of data are used for training\n",
    "\n",
    "X, Y = make_classification(\n",
    "    n_samples=m,\n",
    "    n_features=nx,\n",
    "    n_informative=nx,\n",
    "    n_redundant=0,\n",
    "    n_classes=nlabels,\n",
    "    n_clusters_per_class=1,\n",
    "    class_sep=1,\n",
    "    flip_y=1e-2,\n",
    "    random_state=None,\n",
    ")\n",
    "encoder = OneHotEncoder(sparse_output=False)\n",
    "Y = encoder.fit_transform(Y.reshape(-1, 1))\n",
    "\n",
    "X_train, X_test, Y_train, Y_test = train_test_split(\n",
    "    X, Y, train_size=train_size, random_state=None\n",
    ")\n",
    "m_train = X_train.shape[0]\n",
    "m_test = X_test.shape[0]\n",
    "print(\"m_train\", m_train)\n",
    "print(\"m_test\", m_test)\n",
    "print(\"X train dim\", X_train.shape, \"Y train dim\", Y_train.shape)\n",
    "print(\"X test dim\", X_test.shape, \"Y test dim\", Y_test.shape, \"\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c0445a58-1b59-4e48-84df-2ebe2d233649",
   "metadata": {},
   "source": [
    "## Setup of Model Using Fully Connected Layers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2c3758b9-0960-43d6-987d-dade5ea2d41a",
   "metadata": {},
   "outputs": [],
   "source": [
    "# hyper parameters should be learned as well, however for this toy example\n",
    "# we set them for reasonable computing time and appropriate results\n",
    "epochs = 10\n",
    "no_perceptron_in_hl = np.array([2 * nx, 4 * nx, nlabels])\n",
    "batch_size = 32\n",
    "\n",
    "# model architecture\n",
    "optimizer = keras.optimizers.Adam()\n",
    "loss = keras.losses.CategoricalCrossentropy(\n",
    "    from_logits=False, label_smoothing=0\n",
    ")\n",
    "metrics = [\n",
    "    keras.metrics.CategoricalCrossentropy(),\n",
    "    keras.metrics.CategoricalAccuracy(),\n",
    "]\n",
    "\n",
    "model = keras.Sequential()\n",
    "# apply input layer\n",
    "model.add(keras.Input(shape=(nx,)))\n",
    "# apply hidden layers\n",
    "for n in no_perceptron_in_hl:\n",
    "    model.add(keras.layers.Dense(n, activation=\"relu\"))\n",
    "# apply output layer with softmax for multi-class classificaton\n",
    "model.add(keras.layers.Dense(nlabels, activation=\"softmax\"))\n",
    "# let TF compile the model architecture, one key step in compiling\n",
    "# is to set up the forward and backward propagation workflow through the model\n",
    "model.compile(optimizer=optimizer, loss=loss, metrics=metrics)\n",
    "print(model.summary())\n",
    "# tw = np.sum([K.count_params(w) for w in model.trainable_weights])\n",
    "# print('\\ntrainable_weights', tw, '\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4a52e3f5-7fa6-4912-8f33-dbab76fbf917",
   "metadata": {},
   "source": [
    "## Training of Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "63c19519-624d-4696-a986-0f6ad5880d4c",
   "metadata": {},
   "outputs": [],
   "source": [
    "model.fit(\n",
    "    X_train,\n",
    "    Y_train,\n",
    "    validation_data=(X_test, Y_test),\n",
    "    epochs=epochs,\n",
    "    batch_size=batch_size,\n",
    "    verbose=verbose,\n",
    ")\n",
    "print(model.summary())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "35c47701-5ccb-406e-8f72-8a5bd1e2a101",
   "metadata": {},
   "source": [
    "## Evaluation of Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f26ab89a-b314-4aa2-bd1b-f8bcfae8c8f3",
   "metadata": {},
   "outputs": [],
   "source": [
    "def print_results(X, Y):\n",
    "    # https://stackoverflow.com/questions/48908641/how-to-get-a-single-value-from-softmax-instead-of-probability-get-confusion-ma:\n",
    "    lb = LabelBinarizer()\n",
    "    lb.fit(labels)\n",
    "\n",
    "    m = X.shape[0]\n",
    "    results = model.evaluate(X, Y, batch_size=m, verbose=verbose)\n",
    "    Y_pred = model.predict(X)\n",
    "    cm = tf.math.confusion_matrix(\n",
    "        labels=lb.inverse_transform(Y),\n",
    "        predictions=lb.inverse_transform(Y_pred),\n",
    "        num_classes=nlabels,\n",
    "    )\n",
    "    print(\"data entries\", m)\n",
    "    print(\n",
    "        \"Cost\",\n",
    "        results[0],\n",
    "        \"\\nCategoricalCrossentropy\",\n",
    "        results[1],\n",
    "        \"\\nCategoricalAccuracy\",\n",
    "        results[2],\n",
    "    )\n",
    "    print(\n",
    "        \"nCategoricalAccuracy from Confusion Matrix = \",\n",
    "        np.sum(np.diag(cm.numpy())) / m,\n",
    "    )\n",
    "    print(\"Confusion Matrix in %\\n\", cm / m * 100)\n",
    "\n",
    "\n",
    "print(\"\\n\\nmetrics on train data:\")\n",
    "print_results(X_train, Y_train)\n",
    "\n",
    "print(\"\\n\\nmetrics on !never seen! test data:\")\n",
    "print_results(X_test, Y_test)\n",
    "# recall: the model should generalize well on never before seen data "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "34a66cab-62fd-4554-a88b-56f77fcd0cf1",
   "metadata": {},
   "source": [
    "## Copyright\n",
    "\n",
    "- the notebooks are provided as [Open Educational Resources](https://en.wikipedia.org/wiki/Open_educational_resources)\n",
    "- the text is licensed under [Creative Commons Attribution 4.0](https://creativecommons.org/licenses/by/4.0/)\n",
    "- the code of the IPython examples is licensed under the [MIT license](https://opensource.org/licenses/MIT)\n",
    "- feel free to use the notebooks for your own purposes\n",
    "- please attribute the work as follows: *Frank Schultz, Data Driven Audio Signal Processing - A Tutorial Featuring Computational Examples, University of Rostock* ideally with relevant file(s), github URL https://github.com/spatialaudio/data-driven-audio-signal-processing-exercise, commit number and/or version tag, year."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "myddasp",
   "language": "python",
   "name": "myddasp"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}