CNN_RNN_CONV3d_Gesture_Recognition.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "CNN_RNN_CONV3d_Gesture_Recognition-.ipynb",
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.6.9"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7VpRiJPLlAfI"
      },
      "source": [
        "# Gesture Recognition\n",
        "In this group project, you are going to build a 3D Conv model that will be able to predict the 5 gestures correctly. Please import the following libraries to get started. Once you have completed the code you can download the notebook for making a submission."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "uVzU1JO7lAfO"
      },
      "source": [
        "import numpy as np\n",
        "import os\n",
        "from imageio import imread\n",
        "from skimage.transform import resize\n",
        "import datetime\n",
        "import os\n",
        "import matplotlib.pyplot as plt"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HTHAhJZ0lAfO"
      },
      "source": [
        "We set the random seed so that the results don't vary drastically."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "c6yKRkJylAfP"
      },
      "source": [
        "np.random.seed(30)\n",
        "import random as rn\n",
        "rn.seed(30)\n",
        "from tensorflow import keras\n",
        "import tensorflow as tf\n",
        "tf.random.set_seed(30)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "g7DMxULUlAfP"
      },
      "source": [
        "In this block, you read the folder names for training and validation. You also set the `batch_size` here. Note that you set the batch size in such a way that you are able to use the GPU in full capacity. You keep increasing the batch size until the machine throws an error."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "giA4iW-xlAfQ"
      },
      "source": [
        "**data path: /home/datasets/Project_data**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lPVBc3QolAfQ",
        "outputId": "e6488391-3467-4a45-e3fa-16f26e6cc15d"
      },
      "source": [
        "train_doc = np.random.permutation(open('/home/datasets/Project_data/train.csv').readlines())\n",
        "val_doc = np.random.permutation(open('/home/datasets/Project_data/val.csv').readlines())\n",
        "batch_size = 64 #experiment with the batch size\n",
        "### 256, 128 giving unable to allocate memory while adding layers\n",
        "print(len(train_doc),len(val_doc))"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "663 100\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lgW9xXxNlAfR"
      },
      "source": [
        "## Generator\n",
        "This is one of the most important part of the code. The overall structure of the generator has been given. In the generator, you are going to preprocess the images as you have images of 2 different dimensions as well as create a batch of video frames. You have to experiment with `img_idx`, `y`,`z` and normalization such that you get high accuracy."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "K3GBXIs8lAfS"
      },
      "source": [
        "def generator(source_path, folder_list, batch_size):\n",
        "    print( 'Source path = ', source_path, '; batch size =', batch_size)\n",
        "    global frames,x,y,z\n",
        "    frames = 30\n",
        "    img_idx = [i for i in range(0,frames) if i%2==0]#create a list of image numbers you want to use for a particular video\n",
        "    while True:\n",
        "        t = np.random.permutation(folder_list)\n",
        "        num_batches = len(t)//batch_size# calculate the number of batches\n",
        "        x = len(img_idx)\n",
        "        y,z = 100, 100 ####\n",
        "        for batch in range(num_batches): # we iterate over the number of batches\n",
        "            batch_data = np.zeros((batch_size,x,y,z,3)) # x is the number of images you use for each video, (y,z) is the final size of the input images and 3 is the number of channels RGB\n",
        "            batch_labels = np.zeros((batch_size,5)) # batch_labels is the one hot representation of the output\n",
        "            for folder in range(batch_size): # iterate over the batch_size\n",
        "                imgs = os.listdir(source_path+'/'+ t[folder + (batch*batch_size)].split(';')[0]) # read all the images in the folder\n",
        "                for idx,item in enumerate(img_idx): #  Iterate iver the frames/images of a folder to read them in\n",
        "                    image = imread(source_path+'/'+ t[folder + (batch*batch_size)].strip().split(';')[0]+'/'+imgs[item]).astype(np.float32)\n",
        "#                     print(image.shape)\n",
        "#                     plt.imshow(image/256)\n",
        "#                     return\n",
        "\n",
        "                    if image.shape[0]!=y or image.shape[1]!=z:\n",
        "                        image = resize(image,[y,z], anti_aliasing=True)\n",
        "                    #crop the images and resize them. Note that the images are of 2 different shape \n",
        "                    #and the conv3D will throw error if the inputs in a batch have different shapes\n",
        "                    \n",
        "                    batch_data[folder,idx,:,:,0] = image[:,:,0]/255#normalise and feed in the image\n",
        "                    batch_data[folder,idx,:,:,1] = image[:,:,1]/255#normalise and feed in the image\n",
        "                    batch_data[folder,idx,:,:,2] = image[:,:,2]/255#normalise and feed in the image\n",
        "                    \n",
        "                batch_labels[folder, int(t[folder + (batch*batch_size)].strip().split(';')[2])] = 1\n",
        "            yield batch_data, batch_labels #you yield the batch_data and the batch_labels, remember what does yield do\n",
        "\n",
        "        \n",
        "        # write the code for the remaining data points which are left after full batches\n",
        "        remaining_folders = len(t)%batch_size\n",
        "        if remaining_folders>0:\n",
        "            batch_data = np.zeros((remaining_folders,x,y,z,3)) \n",
        "            batch_labels = np.zeros((remaining_folders,5)) \n",
        "            for folder in range(remaining_folders): \n",
        "                imgs = os.listdir(source_path+'/'+ t[folder + (num_batches*batch_size)].split(';')[0]) # read all the images in the folder\n",
        "                for idx,item in enumerate(img_idx): #  Iterate iver the frames/images of a folder to read them in\n",
        "                    image = imread(source_path+'/'+ t[folder + (num_batches*batch_size)].strip().split(';')[0]+'/'+imgs[item]).astype(np.float32)\n",
        "                    \n",
        "                    if image.shape[0]!=y or image.shape[1]!=z:\n",
        "                        image = resize(image,[y,z], anti_aliasing=True)\n",
        "                    #crop the images and resize them. Note that the images are of 2 different shape \n",
        "                    #and the conv3D will throw error if the inputs in a batch have different shapes\n",
        "                    \n",
        "                    batch_data[folder,idx,:,:,0] = image[:,:,0]/255#normalise and feed in the image\n",
        "                    batch_data[folder,idx,:,:,1] = image[:,:,1]/255#normalise and feed in the image\n",
        "                    batch_data[folder,idx,:,:,2] = image[:,:,2]/255#normalise and feed in the image\n",
        "                    \n",
        "                batch_labels[folder, int(t[folder + (num_batches*batch_size)].strip().split(';')[2])] = 1\n",
        "            yield batch_data, batch_labels \n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "squW_MkdlAfU"
      },
      "source": [
        "# from skimage import data\n",
        "\n",
        "\n",
        "# image = data.coins()\n",
        "# image2 = data.camera()\n",
        "# print(image2.shape)\n",
        "# # plt.imshow(image2)\n",
        "# plt.imshow(resize(image2,[1900,1900], anti_aliasing=True))\n",
        "# plt.show()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "QjmwncZ5lAfV"
      },
      "source": [
        "Note here that a video is represented above in the generator as (number of images, height, width, number of channels). Take this into consideration while creating the model architecture."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YirY9-NxlAfV",
        "outputId": "d9d7301e-1427-47ed-e0cf-6ebf4f7b50cd"
      },
      "source": [
        "curr_dt_time = datetime.datetime.now()\n",
        "train_path = '/home/datasets/Project_data/train'\n",
        "val_path = '/home/datasets/Project_data/val'\n",
        "num_train_sequences = len(train_doc)\n",
        "print('# training sequences =', num_train_sequences)\n",
        "num_val_sequences = len(val_doc)\n",
        "print('# validation sequences =', num_val_sequences)\n",
        "num_epochs = 50 # choose the number of epochs\n",
        "print ('# epochs =', num_epochs)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "# training sequences = 663\n",
            "# validation sequences = 100\n",
            "# epochs = 50\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yMVbbG1wlAfW",
        "outputId": "ebfeb855-e86c-4037-f64e-2b12babea393"
      },
      "source": [
        "# checking images\n",
        "a = generator(train_path, train_doc, batch_size)\n",
        "item = next(a)\n",
        "print(type(item),item[0].shape)\n",
        "# item = next(a)\n",
        "# print(type(item),item[0].shape)\n",
        "# item = next(a)\n",
        "# print(type(item),item[0].shape)\n",
        "# item = next(a)\n",
        "# print(type(item),item[0].shape)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Source path =  /home/datasets/Project_data/train ; batch size = 64\n",
            "<class 'tuple'> (64, 15, 100, 100, 3)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MXYoZqJ8lAfX"
      },
      "source": [
        "## Model\n",
        "Here you make the model using different functionalities that Keras provides. Remember to use `Conv3D` and `MaxPooling3D` and not `Conv2D` and `Maxpooling2D` for a 3D convolution model. You would want to use `TimeDistributed` while building a Conv2D + RNN model. Also remember that the last layer is the softmax. Design the network in such a way that the model is able to give good accuracy on the least number of parameters so that it can fit in the memory of the webcam."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4Ha72FmRlAfX"
      },
      "source": [
        "#######\n",
        "####   Conv3d Model: Accuracy 74/83 (val), batch_size=64\n",
        "#######\n",
        "\n",
        "from tensorflow.keras.models import Sequential\n",
        "from tensorflow.keras.layers import Dense, GRU, Dropout, Flatten, BatchNormalization, Activation, Conv3D, MaxPooling3D\n",
        "from tensorflow.keras.callbacks import ModelCheckpoint, ReduceLROnPlateau\n",
        "from tensorflow.keras.regularizers import L2 #vs l2\n",
        "from tensorflow.keras import optimizers\n",
        "\n",
        "#write your model here\n",
        "model = Sequential()\n",
        "\n",
        "model.add(Conv3D(64, (3, 3, 3) , padding = \"SAME\",\n",
        "                        input_shape=(x, y, z, 3))) #### Large batch_size might not fit params into memory\n",
        "model.add(Activation('relu'))\n",
        "model.add(Conv3D(32, (3, 3, 3), padding = \"SAME\"))\n",
        "model.add(Activation('relu'))\n",
        "model.add(MaxPooling3D(pool_size=(2, 2, 2)))\n",
        "model.add(Dropout(0.5))\n",
        "\n",
        "model.add(Conv3D(16, (3, 3, 3), padding = \"SAME\"))\n",
        "model.add(Activation('relu'))\n",
        "# model.add(Conv3D(16, (3, 3, 3), padding = \"SAME\"))\n",
        "# model.add(Activation('relu'))\n",
        "model.add(MaxPooling3D(pool_size=(2, 2, 2)))\n",
        "model.add(Dropout(0.5))\n",
        "\n",
        "# model.add(Conv3D(32, (3, 3, 3), padding = \"SAME\"))\n",
        "# model.add(Activation('relu'))\n",
        "# model.add(Conv3D(8, (3, 3, 3), padding = \"SAME\"))\n",
        "# model.add(Activation('relu'))\n",
        "# model.add(MaxPooling3D(pool_size=(1, 2, 2)))\n",
        "# model.add(Dropout(0.5))\n",
        "\n",
        "# model.add(Conv3D(32, (3, 3, 3), padding = \"SAME\"))\n",
        "# model.add(Activation('relu'))\n",
        "# model.add(Conv3D(8, (3, 3, 3), padding = \"SAME\"))\n",
        "# model.add(Activation('relu'))\n",
        "# model.add(MaxPooling3D(pool_size=(1, 2, 2)))\n",
        "# model.add(Dropout(0.5))\n",
        "\n",
        "model.add(Flatten())\n",
        "model.add(Dense(512, kernel_regularizer = L2(0.1)))\n",
        "model.add(Activation('relu'))\n",
        "model.add(Dropout(0.5))\n",
        "model.add(Dense(5, kernel_regularizer = L2(0.1)))  #use W_regularizer if .tf.keras.layers.core.Dense\n",
        "model.add(Activation('softmax'))\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "31kx8J7klAfY"
      },
      "source": [
        "Now that you have written the model, the next step is to `compile` the model. When you print the `summary` of the model, you'll see the total number of parameters you have to train."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "BNezF44ZlAfY",
        "scrolled": true,
        "outputId": "67de4fa9-a04e-46c3-eb95-57edc20660bf"
      },
      "source": [
        "learning_rate = 0.001\n",
        "optimiser = optimizers.Adam(lr=learning_rate, clipvalue=1.0)#write your optimizer\n",
        "model.compile(optimizer=optimiser, loss='categorical_crossentropy', metrics=['categorical_accuracy'])\n",
        "print (model.summary())"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"sequential_1\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "conv3d (Conv3D)              (None, 15, 100, 100, 64)  5248      \n",
            "_________________________________________________________________\n",
            "activation (Activation)      (None, 15, 100, 100, 64)  0         \n",
            "_________________________________________________________________\n",
            "conv3d_1 (Conv3D)            (None, 15, 100, 100, 32)  55328     \n",
            "_________________________________________________________________\n",
            "activation_1 (Activation)    (None, 15, 100, 100, 32)  0         \n",
            "_________________________________________________________________\n",
            "max_pooling3d (MaxPooling3D) (None, 7, 50, 50, 32)     0         \n",
            "_________________________________________________________________\n",
            "dropout (Dropout)            (None, 7, 50, 50, 32)     0         \n",
            "_________________________________________________________________\n",
            "conv3d_2 (Conv3D)            (None, 7, 50, 50, 16)     13840     \n",
            "_________________________________________________________________\n",
            "activation_2 (Activation)    (None, 7, 50, 50, 16)     0         \n",
            "_________________________________________________________________\n",
            "max_pooling3d_1 (MaxPooling3 (None, 3, 25, 25, 16)     0         \n",
            "_________________________________________________________________\n",
            "dropout_1 (Dropout)          (None, 3, 25, 25, 16)     0         \n",
            "_________________________________________________________________\n",
            "flatten (Flatten)            (None, 30000)             0         \n",
            "_________________________________________________________________\n",
            "dense (Dense)                (None, 512)               15360512  \n",
            "_________________________________________________________________\n",
            "activation_3 (Activation)    (None, 512)               0         \n",
            "_________________________________________________________________\n",
            "dropout_2 (Dropout)          (None, 512)               0         \n",
            "_________________________________________________________________\n",
            "dense_1 (Dense)              (None, 5)                 2565      \n",
            "_________________________________________________________________\n",
            "activation_4 (Activation)    (None, 5)                 0         \n",
            "=================================================================\n",
            "Total params: 15,437,493\n",
            "Trainable params: 15,437,493\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n",
            "None\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/optimizer_v2/optimizer_v2.py:375: UserWarning: The `lr` argument is deprecated, use `learning_rate` instead.\n",
            "  \"The `lr` argument is deprecated, use `learning_rate` instead.\")\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "UPsPM8eQlAfZ"
      },
      "source": [
        "Let us create the `train_generator` and the `val_generator` which will be used in `.fit_generator`."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HKjBfCNZlAfZ"
      },
      "source": [
        "train_generator = generator(train_path, train_doc, batch_size)\n",
        "val_generator = generator(val_path, val_doc, batch_size)\n",
        "# a = next(val_generator)\n",
        "# print(a[0].shape,type(a[1][0][0]))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ie98VC2xlAfZ"
      },
      "source": [
        "model_name = 'model_init' + '_' + str(curr_dt_time).replace(' ','').replace(':','_') + '/'\n",
        "    \n",
        "if not os.path.exists(model_name):\n",
        "    os.mkdir(model_name)\n",
        "        \n",
        "filepath = model_name + 'model-{epoch:05d}-{loss:.3f}-{categorical_accuracy:.3f}-{val_loss:.3f}-{val_categorical_accuracy:.3f}.h5'\n",
        "\n",
        "checkpoint = ModelCheckpoint(filepath, monitor='val_loss', verbose=1, save_best_only=False, save_weights_only=False, mode='auto',save_freq = \"epoch\")\n",
        "\n",
        "LR = ReduceLROnPlateau(monitor='val_loss', factor=0.2, verbose=1, patience=3)# write the REducelronplateau code here\n",
        "callbacks_list = [checkpoint, LR]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6IX5eYUdlAfa"
      },
      "source": [
        "The `steps_per_epoch` and `validation_steps` are used by `fit` method to decide the number of next() calls it need to make."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VmtybJ7VlAfa"
      },
      "source": [
        "if (num_train_sequences%batch_size) == 0:\n",
        "    steps_per_epoch = int(num_train_sequences/batch_size)\n",
        "else:\n",
        "    steps_per_epoch = (num_train_sequences//batch_size) + 1\n",
        "\n",
        "if (num_val_sequences%batch_size) == 0:\n",
        "    validation_steps = int(num_val_sequences/batch_size)\n",
        "else:\n",
        "    validation_steps = (num_val_sequences//batch_size) + 1"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WobZWGVolAfa"
      },
      "source": [
        "Let us now fit the model. This will start training the model and with the help of the checkpoints, you'll be able to save the model at the end of each epoch."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "kmRAAj1ilAfb",
        "outputId": "e346967f-1fc6-422c-9499-a8251bb8db35"
      },
      "source": [
        "model.fit(train_generator, steps_per_epoch=steps_per_epoch, epochs=num_epochs, verbose=True, \n",
        "                    callbacks=callbacks_list, validation_data=val_generator, \n",
        "                    validation_steps=validation_steps, class_weight=None, workers=1, initial_epoch=0)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Source path =  /home/datasets/Project_data/train ; batch size = 64\n",
            "Epoch 1/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 22.1696 - categorical_accuracy: 0.1946Source path =  /home/datasets/Project_data/val ; batch size = 64\n",
            "\n",
            "Epoch 00001: saving model to model_init_2021-06-2711_18_46.352023/model-00001-22.170-0.195-18.091-0.210.h5\n",
            "11/11 [==============================] - 84s 8s/step - loss: 22.1696 - categorical_accuracy: 0.1946 - val_loss: 18.0909 - val_categorical_accuracy: 0.2100\n",
            "Epoch 2/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 15.5164 - categorical_accuracy: 0.2081\n",
            "Epoch 00002: saving model to model_init_2021-06-2711_18_46.352023/model-00002-15.516-0.208-12.400-0.230.h5\n",
            "11/11 [==============================] - 83s 8s/step - loss: 15.5164 - categorical_accuracy: 0.2081 - val_loss: 12.4001 - val_categorical_accuracy: 0.2300\n",
            "Epoch 3/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 10.5544 - categorical_accuracy: 0.2127\n",
            "Epoch 00003: saving model to model_init_2021-06-2711_18_46.352023/model-00003-10.554-0.213-8.393-0.210.h5\n",
            "11/11 [==============================] - 87s 8s/step - loss: 10.5544 - categorical_accuracy: 0.2127 - val_loss: 8.3934 - val_categorical_accuracy: 0.2100\n",
            "Epoch 4/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 7.1701 - categorical_accuracy: 0.2398\n",
            "Epoch 00004: saving model to model_init_2021-06-2711_18_46.352023/model-00004-7.170-0.240-5.771-0.230.h5\n",
            "11/11 [==============================] - 76s 7s/step - loss: 7.1701 - categorical_accuracy: 0.2398 - val_loss: 5.7712 - val_categorical_accuracy: 0.2300\n",
            "Epoch 5/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 4.9992 - categorical_accuracy: 0.2323\n",
            "Epoch 00005: saving model to model_init_2021-06-2711_18_46.352023/model-00005-4.999-0.232-4.137-0.210.h5\n",
            "11/11 [==============================] - 82s 7s/step - loss: 4.9992 - categorical_accuracy: 0.2323 - val_loss: 4.1371 - val_categorical_accuracy: 0.2100\n",
            "Epoch 6/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 3.6660 - categorical_accuracy: 0.2081\n",
            "Epoch 00006: saving model to model_init_2021-06-2711_18_46.352023/model-00006-3.666-0.208-3.156-0.210.h5\n",
            "11/11 [==============================] - 79s 7s/step - loss: 3.6660 - categorical_accuracy: 0.2081 - val_loss: 3.1559 - val_categorical_accuracy: 0.2100\n",
            "Epoch 7/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 2.8853 - categorical_accuracy: 0.1991\n",
            "Epoch 00007: saving model to model_init_2021-06-2711_18_46.352023/model-00007-2.885-0.199-2.588-0.210.h5\n",
            "11/11 [==============================] - 83s 8s/step - loss: 2.8853 - categorical_accuracy: 0.1991 - val_loss: 2.5885 - val_categorical_accuracy: 0.2100\n",
            "Epoch 8/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 2.4231 - categorical_accuracy: 0.1840\n",
            "Epoch 00008: saving model to model_init_2021-06-2711_18_46.352023/model-00008-2.423-0.184-2.252-0.220.h5\n",
            "11/11 [==============================] - 80s 7s/step - loss: 2.4231 - categorical_accuracy: 0.1840 - val_loss: 2.2517 - val_categorical_accuracy: 0.2200\n",
            "Epoch 9/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 2.1465 - categorical_accuracy: 0.2112\n",
            "Epoch 00009: saving model to model_init_2021-06-2711_18_46.352023/model-00009-2.146-0.211-2.052-0.210.h5\n",
            "11/11 [==============================] - 86s 8s/step - loss: 2.1465 - categorical_accuracy: 0.2112 - val_loss: 2.0516 - val_categorical_accuracy: 0.2100\n",
            "Epoch 10/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 1.9892 - categorical_accuracy: 0.2142\n",
            "Epoch 00010: saving model to model_init_2021-06-2711_18_46.352023/model-00010-1.989-0.214-1.938-0.250.h5\n",
            "11/11 [==============================] - 81s 7s/step - loss: 1.9892 - categorical_accuracy: 0.2142 - val_loss: 1.9380 - val_categorical_accuracy: 0.2500\n",
            "Epoch 11/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 1.8970 - categorical_accuracy: 0.2217\n",
            "Epoch 00011: saving model to model_init_2021-06-2711_18_46.352023/model-00011-1.897-0.222-1.862-0.290.h5\n",
            "11/11 [==============================] - 91s 8s/step - loss: 1.8970 - categorical_accuracy: 0.2217 - val_loss: 1.8617 - val_categorical_accuracy: 0.2900\n",
            "Epoch 12/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 1.8226 - categorical_accuracy: 0.2594\n",
            "Epoch 00012: saving model to model_init_2021-06-2711_18_46.352023/model-00012-1.823-0.259-1.796-0.410.h5\n",
            "11/11 [==============================] - 75s 7s/step - loss: 1.8226 - categorical_accuracy: 0.2594 - val_loss: 1.7960 - val_categorical_accuracy: 0.4100\n",
            "Epoch 13/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 1.7651 - categorical_accuracy: 0.2413\n",
            "Epoch 00013: saving model to model_init_2021-06-2711_18_46.352023/model-00013-1.765-0.241-1.737-0.300.h5\n",
            "11/11 [==============================] - 79s 7s/step - loss: 1.7651 - categorical_accuracy: 0.2413 - val_loss: 1.7374 - val_categorical_accuracy: 0.3000\n",
            "Epoch 14/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 1.7099 - categorical_accuracy: 0.3047\n",
            "Epoch 00014: saving model to model_init_2021-06-2711_18_46.352023/model-00014-1.710-0.305-1.710-0.410.h5\n",
            "11/11 [==============================] - 80s 7s/step - loss: 1.7099 - categorical_accuracy: 0.3047 - val_loss: 1.7102 - val_categorical_accuracy: 0.4100\n",
            "Epoch 15/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 1.6581 - categorical_accuracy: 0.3258\n",
            "Epoch 00015: saving model to model_init_2021-06-2711_18_46.352023/model-00015-1.658-0.326-1.624-0.460.h5\n",
            "11/11 [==============================] - 81s 7s/step - loss: 1.6581 - categorical_accuracy: 0.3258 - val_loss: 1.6243 - val_categorical_accuracy: 0.4600\n",
            "Epoch 16/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 1.5834 - categorical_accuracy: 0.4027\n",
            "Epoch 00016: saving model to model_init_2021-06-2711_18_46.352023/model-00016-1.583-0.403-1.457-0.490.h5\n",
            "11/11 [==============================] - 90s 8s/step - loss: 1.5834 - categorical_accuracy: 0.4027 - val_loss: 1.4573 - val_categorical_accuracy: 0.4900\n",
            "Epoch 17/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 1.5344 - categorical_accuracy: 0.4178\n",
            "Epoch 00017: saving model to model_init_2021-06-2711_18_46.352023/model-00017-1.534-0.418-1.494-0.510.h5\n",
            "11/11 [==============================] - 78s 7s/step - loss: 1.5344 - categorical_accuracy: 0.4178 - val_loss: 1.4937 - val_categorical_accuracy: 0.5100\n",
            "Epoch 18/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 1.4709 - categorical_accuracy: 0.4510\n",
            "Epoch 00018: saving model to model_init_2021-06-2711_18_46.352023/model-00018-1.471-0.451-1.345-0.490.h5\n",
            "11/11 [==============================] - 73s 7s/step - loss: 1.4709 - categorical_accuracy: 0.4510 - val_loss: 1.3445 - val_categorical_accuracy: 0.4900\n",
            "Epoch 19/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 1.4550 - categorical_accuracy: 0.4691\n",
            "Epoch 00019: saving model to model_init_2021-06-2711_18_46.352023/model-00019-1.455-0.469-1.387-0.580.h5\n",
            "11/11 [==============================] - 89s 8s/step - loss: 1.4550 - categorical_accuracy: 0.4691 - val_loss: 1.3873 - val_categorical_accuracy: 0.5800\n",
            "Epoch 20/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 1.4208 - categorical_accuracy: 0.4766\n",
            "Epoch 00020: saving model to model_init_2021-06-2711_18_46.352023/model-00020-1.421-0.477-1.399-0.520.h5\n",
            "11/11 [==============================] - 81s 7s/step - loss: 1.4208 - categorical_accuracy: 0.4766 - val_loss: 1.3989 - val_categorical_accuracy: 0.5200\n",
            "Epoch 21/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 1.3514 - categorical_accuracy: 0.5249\n",
            "Epoch 00021: saving model to model_init_2021-06-2711_18_46.352023/model-00021-1.351-0.525-1.294-0.620.h5\n",
            "11/11 [==============================] - 88s 8s/step - loss: 1.3514 - categorical_accuracy: 0.5249 - val_loss: 1.2941 - val_categorical_accuracy: 0.6200\n",
            "Epoch 22/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 1.3316 - categorical_accuracy: 0.5370\n",
            "Epoch 00022: saving model to model_init_2021-06-2711_18_46.352023/model-00022-1.332-0.537-1.262-0.620.h5\n",
            "11/11 [==============================] - 81s 7s/step - loss: 1.3316 - categorical_accuracy: 0.5370 - val_loss: 1.2620 - val_categorical_accuracy: 0.6200\n",
            "Epoch 23/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 1.3005 - categorical_accuracy: 0.5566\n",
            "Epoch 00023: saving model to model_init_2021-06-2711_18_46.352023/model-00023-1.300-0.557-1.292-0.600.h5\n",
            "11/11 [==============================] - 81s 7s/step - loss: 1.3005 - categorical_accuracy: 0.5566 - val_loss: 1.2924 - val_categorical_accuracy: 0.6000\n",
            "Epoch 24/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 1.2868 - categorical_accuracy: 0.5852\n",
            "Epoch 00024: saving model to model_init_2021-06-2711_18_46.352023/model-00024-1.287-0.585-1.309-0.640.h5\n",
            "11/11 [==============================] - 79s 7s/step - loss: 1.2868 - categorical_accuracy: 0.5852 - val_loss: 1.3094 - val_categorical_accuracy: 0.6400\n",
            "Epoch 25/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 1.2612 - categorical_accuracy: 0.5867\n",
            "Epoch 00025: saving model to model_init_2021-06-2711_18_46.352023/model-00025-1.261-0.587-1.255-0.650.h5\n",
            "11/11 [==============================] - 85s 8s/step - loss: 1.2612 - categorical_accuracy: 0.5867 - val_loss: 1.2547 - val_categorical_accuracy: 0.6500\n",
            "Epoch 26/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 1.2720 - categorical_accuracy: 0.6139\n",
            "Epoch 00026: saving model to model_init_2021-06-2711_18_46.352023/model-00026-1.272-0.614-1.188-0.750.h5\n",
            "11/11 [==============================] - 77s 7s/step - loss: 1.2720 - categorical_accuracy: 0.6139 - val_loss: 1.1879 - val_categorical_accuracy: 0.7500\n",
            "Epoch 27/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 1.2247 - categorical_accuracy: 0.6154\n",
            "Epoch 00027: saving model to model_init_2021-06-2711_18_46.352023/model-00027-1.225-0.615-1.259-0.730.h5\n",
            "11/11 [==============================] - 72s 7s/step - loss: 1.2247 - categorical_accuracy: 0.6154 - val_loss: 1.2592 - val_categorical_accuracy: 0.7300\n",
            "Epoch 28/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 1.1645 - categorical_accuracy: 0.6305\n",
            "Epoch 00028: saving model to model_init_2021-06-2711_18_46.352023/model-00028-1.165-0.630-1.108-0.710.h5\n",
            "11/11 [==============================] - 70s 6s/step - loss: 1.1645 - categorical_accuracy: 0.6305 - val_loss: 1.1085 - val_categorical_accuracy: 0.7100\n",
            "Epoch 29/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 1.1538 - categorical_accuracy: 0.6516\n",
            "Epoch 00029: saving model to model_init_2021-06-2711_18_46.352023/model-00029-1.154-0.652-1.084-0.740.h5\n",
            "11/11 [==============================] - 90s 8s/step - loss: 1.1538 - categorical_accuracy: 0.6516 - val_loss: 1.0842 - val_categorical_accuracy: 0.7400\n",
            "Epoch 30/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 1.1166 - categorical_accuracy: 0.6652\n",
            "Epoch 00030: saving model to model_init_2021-06-2711_18_46.352023/model-00030-1.117-0.665-0.992-0.770.h5\n",
            "11/11 [==============================] - 91s 8s/step - loss: 1.1166 - categorical_accuracy: 0.6652 - val_loss: 0.9919 - val_categorical_accuracy: 0.7700\n",
            "Epoch 31/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 1.1379 - categorical_accuracy: 0.6802\n",
            "Epoch 00031: saving model to model_init_2021-06-2711_18_46.352023/model-00031-1.138-0.680-1.098-0.680.h5\n",
            "11/11 [==============================] - 91s 8s/step - loss: 1.1379 - categorical_accuracy: 0.6802 - val_loss: 1.0980 - val_categorical_accuracy: 0.6800\n",
            "Epoch 32/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 1.2325 - categorical_accuracy: 0.6365\n",
            "Epoch 00032: saving model to model_init_2021-06-2711_18_46.352023/model-00032-1.232-0.637-1.286-0.650.h5\n",
            "11/11 [==============================] - 93s 8s/step - loss: 1.2325 - categorical_accuracy: 0.6365 - val_loss: 1.2857 - val_categorical_accuracy: 0.6500\n",
            "Epoch 33/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 1.1526 - categorical_accuracy: 0.6637\n",
            "Epoch 00033: saving model to model_init_2021-06-2711_18_46.352023/model-00033-1.153-0.664-1.056-0.760.h5\n",
            "\n",
            "Epoch 00033: ReduceLROnPlateau reducing learning rate to 0.00020000000949949026.\n",
            "11/11 [==============================] - 90s 8s/step - loss: 1.1526 - categorical_accuracy: 0.6637 - val_loss: 1.0562 - val_categorical_accuracy: 0.7600\n",
            "Epoch 34/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 1.0733 - categorical_accuracy: 0.6742\n",
            "Epoch 00034: saving model to model_init_2021-06-2711_18_46.352023/model-00034-1.073-0.674-1.038-0.740.h5\n",
            "11/11 [==============================] - 80s 7s/step - loss: 1.0733 - categorical_accuracy: 0.6742 - val_loss: 1.0379 - val_categorical_accuracy: 0.7400\n",
            "Epoch 35/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 1.0670 - categorical_accuracy: 0.6893\n",
            "Epoch 00035: saving model to model_init_2021-06-2711_18_46.352023/model-00035-1.067-0.689-1.003-0.770.h5\n",
            "11/11 [==============================] - 82s 7s/step - loss: 1.0670 - categorical_accuracy: 0.6893 - val_loss: 1.0034 - val_categorical_accuracy: 0.7700\n",
            "Epoch 36/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 0.9899 - categorical_accuracy: 0.7104\n",
            "Epoch 00036: saving model to model_init_2021-06-2711_18_46.352023/model-00036-0.990-0.710-0.987-0.740.h5\n",
            "11/11 [==============================] - 88s 8s/step - loss: 0.9899 - categorical_accuracy: 0.7104 - val_loss: 0.9871 - val_categorical_accuracy: 0.7400\n",
            "Epoch 37/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 0.9401 - categorical_accuracy: 0.7421\n",
            "Epoch 00037: saving model to model_init_2021-06-2711_18_46.352023/model-00037-0.940-0.742-0.935-0.810.h5\n",
            "11/11 [==============================] - 83s 8s/step - loss: 0.9401 - categorical_accuracy: 0.7421 - val_loss: 0.9352 - val_categorical_accuracy: 0.8100\n",
            "Epoch 38/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 0.9310 - categorical_accuracy: 0.7270\n",
            "Epoch 00038: saving model to model_init_2021-06-2711_18_46.352023/model-00038-0.931-0.727-0.922-0.790.h5\n",
            "11/11 [==============================] - 79s 7s/step - loss: 0.9310 - categorical_accuracy: 0.7270 - val_loss: 0.9215 - val_categorical_accuracy: 0.7900\n",
            "Epoch 39/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 0.9181 - categorical_accuracy: 0.7360\n",
            "Epoch 00039: saving model to model_init_2021-06-2711_18_46.352023/model-00039-0.918-0.736-0.901-0.800.h5\n",
            "11/11 [==============================] - 83s 8s/step - loss: 0.9181 - categorical_accuracy: 0.7360 - val_loss: 0.9007 - val_categorical_accuracy: 0.8000\n",
            "Epoch 40/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 0.8985 - categorical_accuracy: 0.7496\n",
            "Epoch 00040: saving model to model_init_2021-06-2711_18_46.352023/model-00040-0.899-0.750-0.831-0.830.h5\n",
            "11/11 [==============================] - 68s 6s/step - loss: 0.8985 - categorical_accuracy: 0.7496 - val_loss: 0.8315 - val_categorical_accuracy: 0.8300\n",
            "Epoch 41/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 0.8800 - categorical_accuracy: 0.7541\n",
            "Epoch 00041: saving model to model_init_2021-06-2711_18_46.352023/model-00041-0.880-0.754-0.854-0.820.h5\n",
            "11/11 [==============================] - 71s 6s/step - loss: 0.8800 - categorical_accuracy: 0.7541 - val_loss: 0.8540 - val_categorical_accuracy: 0.8200\n",
            "Epoch 42/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 0.8967 - categorical_accuracy: 0.7255\n",
            "Epoch 00042: saving model to model_init_2021-06-2711_18_46.352023/model-00042-0.897-0.725-0.853-0.840.h5\n",
            "11/11 [==============================] - 68s 6s/step - loss: 0.8967 - categorical_accuracy: 0.7255 - val_loss: 0.8534 - val_categorical_accuracy: 0.8400\n",
            "Epoch 43/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 0.8876 - categorical_accuracy: 0.7481\n",
            "Epoch 00043: saving model to model_init_2021-06-2711_18_46.352023/model-00043-0.888-0.748-0.852-0.770.h5\n",
            "\n",
            "Epoch 00043: ReduceLROnPlateau reducing learning rate to 4.0000001899898055e-05.\n",
            "11/11 [==============================] - 71s 6s/step - loss: 0.8876 - categorical_accuracy: 0.7481 - val_loss: 0.8517 - val_categorical_accuracy: 0.7700\n",
            "Epoch 44/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 0.8489 - categorical_accuracy: 0.7511\n",
            "Epoch 00044: saving model to model_init_2021-06-2711_18_46.352023/model-00044-0.849-0.751-0.891-0.760.h5\n",
            "11/11 [==============================] - 69s 6s/step - loss: 0.8489 - categorical_accuracy: 0.7511 - val_loss: 0.8909 - val_categorical_accuracy: 0.7600\n",
            "Epoch 45/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 0.8295 - categorical_accuracy: 0.7496\n",
            "Epoch 00045: saving model to model_init_2021-06-2711_18_46.352023/model-00045-0.830-0.750-0.828-0.800.h5\n",
            "11/11 [==============================] - 71s 6s/step - loss: 0.8295 - categorical_accuracy: 0.7496 - val_loss: 0.8276 - val_categorical_accuracy: 0.8000\n",
            "Epoch 46/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 0.8454 - categorical_accuracy: 0.7421\n",
            "Epoch 00046: saving model to model_init_2021-06-2711_18_46.352023/model-00046-0.845-0.742-0.865-0.760.h5\n",
            "11/11 [==============================] - 69s 6s/step - loss: 0.8454 - categorical_accuracy: 0.7421 - val_loss: 0.8652 - val_categorical_accuracy: 0.7600\n",
            "Epoch 47/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 0.8194 - categorical_accuracy: 0.7481\n",
            "Epoch 00047: saving model to model_init_2021-06-2711_18_46.352023/model-00047-0.819-0.748-0.824-0.820.h5\n",
            "11/11 [==============================] - 71s 6s/step - loss: 0.8194 - categorical_accuracy: 0.7481 - val_loss: 0.8239 - val_categorical_accuracy: 0.8200\n",
            "Epoch 48/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 0.8115 - categorical_accuracy: 0.7541\n",
            "Epoch 00048: saving model to model_init_2021-06-2711_18_46.352023/model-00048-0.812-0.754-0.789-0.810.h5\n",
            "11/11 [==============================] - 69s 6s/step - loss: 0.8115 - categorical_accuracy: 0.7541 - val_loss: 0.7886 - val_categorical_accuracy: 0.8100\n",
            "Epoch 49/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 0.8175 - categorical_accuracy: 0.7406\n",
            "Epoch 00049: saving model to model_init_2021-06-2711_18_46.352023/model-00049-0.817-0.741-0.812-0.820.h5\n",
            "11/11 [==============================] - 72s 7s/step - loss: 0.8175 - categorical_accuracy: 0.7406 - val_loss: 0.8119 - val_categorical_accuracy: 0.8200\n",
            "Epoch 50/50\n",
            "11/11 [==============================] - ETA: 0s - loss: 0.8508 - categorical_accuracy: 0.7406\n",
            "Epoch 00050: saving model to model_init_2021-06-2711_18_46.352023/model-00050-0.851-0.741-0.753-0.830.h5\n",
            "11/11 [==============================] - 70s 6s/step - loss: 0.8508 - categorical_accuracy: 0.7406 - val_loss: 0.7529 - val_categorical_accuracy: 0.8300\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tensorflow.python.keras.callbacks.History at 0x7f8bb76629e8>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 13
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Zx3pt_dOlAfb",
        "outputId": "8e0d3943-5385-47c4-d60a-39e7ea97e198"
      },
      "source": [
        "# !ls -ltrh /home/dl_content/'Upgrad DL'/'Gesture Recognition Assignment'/model_init_2021-*\n",
        "# !du -ah /home | grep -v \"/$\" | sort -rh | head -6 \n",
        "# !rm -rf /home/.local/share/Trash/files\n",
        "# !df -h"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Filesystem      Size  Used Avail Use% Mounted on\n",
            "overlay          10G   92K   10G   1% /\n",
            "tmpfs            64M     0   64M   0% /dev\n",
            "tmpfs           126G     0  126G   0% /sys/fs/cgroup\n",
            "/dev/nvme0n1     10G  5.8G  4.3G  58% /datasets\n",
            "tmpfs           126G  720K  126G   1% /dev/shm\n",
            "tmpfs           126G   12K  126G   1% /proc/driver/nvidia\n",
            "/dev/sda2       3.5T  395G  2.9T  12% /usr/bin/nvidia-smi\n",
            "tmpfs            26G  4.1M   26G   1% /run/nvidia-persistenced/socket\n",
            "udev            126G     0  126G   0% /dev/nvidia0\n",
            "tmpfs           126G     0  126G   0% /proc/asound\n",
            "tmpfs           126G     0  126G   0% /proc/acpi\n",
            "tmpfs           126G     0  126G   0% /proc/scsi\n",
            "tmpfs           126G     0  126G   0% /sys/firmware\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nLqpDCLklAfc"
      },
      "source": [
        "### CNN-RNN Model"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "t-BOwI14lAfc"
      },
      "source": [
        "########\n",
        "####   CNN-RNN Model: Accuray 94/82(val) , batch_size = 32\n",
        "########\n",
        "\n",
        "\n",
        "from tensorflow.keras.models import Sequential\n",
        "from tensorflow.keras.layers import Dense,TimeDistributed, LSTM, GRU, Dropout, Flatten, BatchNormalization, Activation, Conv2D,Conv3D, MaxPooling2D, MaxPooling3D\n",
        "from tensorflow.keras.callbacks import ModelCheckpoint, ReduceLROnPlateau\n",
        "from tensorflow.keras.regularizers import L2 #vs l2\n",
        "from tensorflow.keras import optimizers\n",
        "\n",
        "#write your model here\n",
        "model = Sequential()\n",
        "\n",
        "## TimeDistributed automaticaly incorporates x=15, when model .build\n",
        "\n",
        "model.add(TimeDistributed(Conv2D(32, (3, 3) , padding = \"SAME\",\n",
        "                        input_shape=(y, z, 3)))) #### Large batch_size might not fit params into memory\n",
        "model.add(TimeDistributed(Activation('relu')))\n",
        "model.add(TimeDistributed(Conv2D(32, ( 3, 3), padding = \"SAME\")))\n",
        "model.add(TimeDistributed(Activation('relu')))\n",
        "model.add(TimeDistributed(MaxPooling2D(pool_size=(2, 2))))\n",
        "model.add(TimeDistributed(Dropout(0.5)))\n",
        "\n",
        "# model.add(TimeDistributed(Conv2D(64, (3, 3) , padding = \"SAME\"))) \n",
        "# model.add(TimeDistributed(Activation('relu')))\n",
        "# model.add(TimeDistributed(Conv2D(64, ( 3, 3), padding = \"SAME\")))\n",
        "# model.add(TimeDistributed(Activation('relu')))\n",
        "# model.add(TimeDistributed(MaxPooling2D(pool_size=(2, 2))))\n",
        "# model.add(TimeDistributed(Dropout(0.5)))\n",
        "\n",
        "# model.add(TimeDistributed(Conv2D(32, (3, 3) , padding = \"SAME\"))) \n",
        "# model.add(TimeDistributed(Activation('relu')))\n",
        "# model.add(TimeDistributed(Conv2D(32, ( 3, 3), padding = \"SAME\")))\n",
        "# model.add(TimeDistributed(Activation('relu')))\n",
        "# model.add(TimeDistributed(MaxPooling2D(pool_size=(2, 2))))\n",
        "# model.add(TimeDistributed(Dropout(0.5)))\n",
        "\n",
        "model.add(TimeDistributed(Flatten()))\n",
        "model.add(LSTM(32, \n",
        "#                input_shape = [None,x]\n",
        "#                 activation=\"relu\",\n",
        "#                 dropout = 0.5\n",
        "              ))    \n",
        "model.add(Dropout(0.5))\n",
        "\n",
        "## number of params - 74240 = 4 set of wts * ( 128 output  * ( 16 emb inputdim + 1 bias) +   128 output^2)\n",
        "\n",
        "# Layer 5 - Dense FFN with Dropout\n",
        "model.add(BatchNormalization())\n",
        "model.add(Dense(128))\n",
        "model.add(Dropout(0.5))\n",
        "# model.add(BatchNormalization())\n",
        "\n",
        "model.add(Dense(5, \n",
        "#                 kernel_initializer = weights,\n",
        "                    name='output'))\n",
        "model.add(Activation('softmax'))\n",
        "\n",
        "\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "VC8iPMxVlAfc",
        "outputId": "8fa1d1f7-5b7b-4818-c904-812f250d68f8"
      },
      "source": [
        "learning_rate = 0.001\n",
        "optimiser = optimizers.Adam(lr=learning_rate, clipvalue=1.0)#write your optimizer\n",
        "model.compile(optimizer=optimiser, loss='categorical_crossentropy', metrics=['categorical_accuracy'])\n",
        "\n",
        "model.build(input_shape = [None, x ,y, z ,3]) ##### specify seq\n",
        "\n",
        "print (model.summary())"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"sequential\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "time_distributed (TimeDistri (None, 15, 100, 100, 32)  896       \n",
            "_________________________________________________________________\n",
            "time_distributed_1 (TimeDist (None, 15, 100, 100, 32)  0         \n",
            "_________________________________________________________________\n",
            "time_distributed_2 (TimeDist (None, 15, 100, 100, 32)  9248      \n",
            "_________________________________________________________________\n",
            "time_distributed_3 (TimeDist (None, 15, 100, 100, 32)  0         \n",
            "_________________________________________________________________\n",
            "time_distributed_4 (TimeDist (None, 15, 50, 50, 32)    0         \n",
            "_________________________________________________________________\n",
            "time_distributed_5 (TimeDist (None, 15, 50, 50, 32)    0         \n",
            "_________________________________________________________________\n",
            "time_distributed_6 (TimeDist (None, 15, 80000)         0         \n",
            "_________________________________________________________________\n",
            "lstm (LSTM)                  (None, 32)                10244224  \n",
            "_________________________________________________________________\n",
            "dropout_1 (Dropout)          (None, 32)                0         \n",
            "_________________________________________________________________\n",
            "batch_normalization (BatchNo (None, 32)                128       \n",
            "_________________________________________________________________\n",
            "dense (Dense)                (None, 128)               4224      \n",
            "_________________________________________________________________\n",
            "dropout_2 (Dropout)          (None, 128)               0         \n",
            "_________________________________________________________________\n",
            "output (Dense)               (None, 5)                 645       \n",
            "_________________________________________________________________\n",
            "activation_2 (Activation)    (None, 5)                 0         \n",
            "=================================================================\n",
            "Total params: 10,259,365\n",
            "Trainable params: 10,259,301\n",
            "Non-trainable params: 64\n",
            "_________________________________________________________________\n",
            "None\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "i2gMVnLAlAfd"
      },
      "source": [
        "batch_size = 32"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MaTF0UQKlAfd"
      },
      "source": [
        "# train_ds = tf.data.Dataset.from_generator(train_generator.generate, output_types=(tf.float32, tf.int32))\n",
        "# train_ds = train_ds.map(lambda x,y : (x,y)).prefetch(buffer_size=1000)\n",
        "# train_ds = train_ds.batch(batch_size)\n",
        "\n",
        "# val_ds = tf.data.Dataset.from_generator(val_generator.generate, output_types=(tf.float32, tf.int32))\n",
        "# val_ds = val_ds.map(lambda x,y : (x,y)).prefetch(buffer_size=1000)\n",
        "# # dataset = dataset.map(lambda x,y : (x,y), num_parallel_calls=NUM_THREADS).prefetch(buffer_size=1000)\n",
        "# val_ds = val_ds.batch(batch_size)\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WrAryJRVlAfd"
      },
      "source": [
        "train_generator = generator(train_path, train_doc, batch_size)\n",
        "val_generator = generator(val_path, val_doc, batch_size)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "cXaKX2XllAfe"
      },
      "source": [
        "model_name = 'model_init' + '_' + str(curr_dt_time).replace(' ','').replace(':','_') + '/'\n",
        "    \n",
        "if not os.path.exists(model_name):\n",
        "    os.mkdir(model_name)\n",
        "        \n",
        "filepath = model_name + 'model-{epoch:05d}-{loss:.3f}-{categorical_accuracy:.3f}-{val_loss:.3f}-{val_categorical_accuracy:.3f}.h5'\n",
        "\n",
        "checkpoint = ModelCheckpoint(filepath, monitor='val_loss', verbose=1, save_best_only=False, save_weights_only=False, mode='auto',save_freq = \"epoch\")\n",
        "\n",
        "LR = ReduceLROnPlateau(monitor='val_loss', factor=0.2, verbose=1, patience=3)# write the REducelronplateau code here\n",
        "callbacks_list = [checkpoint, LR]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LLOhAXBTlAfe"
      },
      "source": [
        "if (num_train_sequences%batch_size) == 0:\n",
        "    steps_per_epoch = int(num_train_sequences/batch_size)\n",
        "else:\n",
        "    steps_per_epoch = (num_train_sequences//batch_size) + 1\n",
        "\n",
        "if (num_val_sequences%batch_size) == 0:\n",
        "    validation_steps = int(num_val_sequences/batch_size)\n",
        "else:\n",
        "    validation_steps = (num_val_sequences//batch_size) + 1"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_JLWh3mS6tH8",
        "outputId": "d7012634-af98-4195-9a2d-a93250eb7307"
      },
      "source": [
        "model.fit(train_generator, steps_per_epoch=steps_per_epoch, epochs=num_epochs, verbose=True, \n",
        "                    callbacks=callbacks_list, validation_data=val_generator, \n",
        "                    validation_steps=validation_steps, class_weight=None, workers=1, initial_epoch=0)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Source path =  /home/datasets/Project_data/train ; batch size = 32\n",
            "Epoch 1/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 1.8293 - categorical_accuracy: 0.2428Source path =  /home/datasets/Project_data/val ; batch size = 32\n",
            "\n",
            "Epoch 00001: saving model to model_init_2021-06-2718_19_46.453166/model-00001-1.829-0.243-1.604-0.180.h5\n",
            "21/21 [==============================] - 70s 3s/step - loss: 1.8293 - categorical_accuracy: 0.2428 - val_loss: 1.6037 - val_categorical_accuracy: 0.1800\n",
            "Epoch 2/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 1.6770 - categorical_accuracy: 0.2172\n",
            "Epoch 00002: saving model to model_init_2021-06-2718_19_46.453166/model-00002-1.677-0.217-1.597-0.210.h5\n",
            "21/21 [==============================] - 72s 3s/step - loss: 1.6770 - categorical_accuracy: 0.2172 - val_loss: 1.5970 - val_categorical_accuracy: 0.2100\n",
            "Epoch 3/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 1.6021 - categorical_accuracy: 0.2353\n",
            "Epoch 00003: saving model to model_init_2021-06-2718_19_46.453166/model-00003-1.602-0.235-1.573-0.320.h5\n",
            "21/21 [==============================] - 70s 3s/step - loss: 1.6021 - categorical_accuracy: 0.2353 - val_loss: 1.5732 - val_categorical_accuracy: 0.3200\n",
            "Epoch 4/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 1.5524 - categorical_accuracy: 0.2836\n",
            "Epoch 00004: saving model to model_init_2021-06-2718_19_46.453166/model-00004-1.552-0.284-1.581-0.280.h5\n",
            "21/21 [==============================] - 68s 3s/step - loss: 1.5524 - categorical_accuracy: 0.2836 - val_loss: 1.5805 - val_categorical_accuracy: 0.2800\n",
            "Epoch 5/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 1.4889 - categorical_accuracy: 0.3243\n",
            "Epoch 00005: saving model to model_init_2021-06-2718_19_46.453166/model-00005-1.489-0.324-1.526-0.490.h5\n",
            "21/21 [==============================] - 71s 3s/step - loss: 1.4889 - categorical_accuracy: 0.3243 - val_loss: 1.5257 - val_categorical_accuracy: 0.4900\n",
            "Epoch 6/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 1.3662 - categorical_accuracy: 0.4238\n",
            "Epoch 00006: saving model to model_init_2021-06-2718_19_46.453166/model-00006-1.366-0.424-1.402-0.600.h5\n",
            "21/21 [==============================] - 71s 3s/step - loss: 1.3662 - categorical_accuracy: 0.4238 - val_loss: 1.4022 - val_categorical_accuracy: 0.6000\n",
            "Epoch 7/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 1.1814 - categorical_accuracy: 0.5309\n",
            "Epoch 00007: saving model to model_init_2021-06-2718_19_46.453166/model-00007-1.181-0.531-1.458-0.500.h5\n",
            "21/21 [==============================] - 70s 3s/step - loss: 1.1814 - categorical_accuracy: 0.5309 - val_loss: 1.4582 - val_categorical_accuracy: 0.5000\n",
            "Epoch 8/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 1.0703 - categorical_accuracy: 0.5822\n",
            "Epoch 00008: saving model to model_init_2021-06-2718_19_46.453166/model-00008-1.070-0.582-1.322-0.580.h5\n",
            "21/21 [==============================] - 67s 3s/step - loss: 1.0703 - categorical_accuracy: 0.5822 - val_loss: 1.3223 - val_categorical_accuracy: 0.5800\n",
            "Epoch 9/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.9965 - categorical_accuracy: 0.6094\n",
            "Epoch 00009: saving model to model_init_2021-06-2718_19_46.453166/model-00009-0.997-0.609-1.110-0.690.h5\n",
            "21/21 [==============================] - 70s 3s/step - loss: 0.9965 - categorical_accuracy: 0.6094 - val_loss: 1.1096 - val_categorical_accuracy: 0.6900\n",
            "Epoch 10/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.8465 - categorical_accuracy: 0.6923\n",
            "Epoch 00010: saving model to model_init_2021-06-2718_19_46.453166/model-00010-0.847-0.692-1.051-0.680.h5\n",
            "21/21 [==============================] - 70s 3s/step - loss: 0.8465 - categorical_accuracy: 0.6923 - val_loss: 1.0513 - val_categorical_accuracy: 0.6800\n",
            "Epoch 11/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.7256 - categorical_accuracy: 0.7466\n",
            "Epoch 00011: saving model to model_init_2021-06-2718_19_46.453166/model-00011-0.726-0.747-0.918-0.650.h5\n",
            "21/21 [==============================] - 71s 3s/step - loss: 0.7256 - categorical_accuracy: 0.7466 - val_loss: 0.9184 - val_categorical_accuracy: 0.6500\n",
            "Epoch 12/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.6284 - categorical_accuracy: 0.7647\n",
            "Epoch 00012: saving model to model_init_2021-06-2718_19_46.453166/model-00012-0.628-0.765-0.905-0.730.h5\n",
            "21/21 [==============================] - 68s 3s/step - loss: 0.6284 - categorical_accuracy: 0.7647 - val_loss: 0.9052 - val_categorical_accuracy: 0.7300\n",
            "Epoch 13/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.5410 - categorical_accuracy: 0.8024\n",
            "Epoch 00013: saving model to model_init_2021-06-2718_19_46.453166/model-00013-0.541-0.802-0.751-0.790.h5\n",
            "21/21 [==============================] - 71s 3s/step - loss: 0.5410 - categorical_accuracy: 0.8024 - val_loss: 0.7508 - val_categorical_accuracy: 0.7900\n",
            "Epoch 14/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.4603 - categorical_accuracy: 0.8371\n",
            "Epoch 00014: saving model to model_init_2021-06-2718_19_46.453166/model-00014-0.460-0.837-0.991-0.700.h5\n",
            "21/21 [==============================] - 70s 3s/step - loss: 0.4603 - categorical_accuracy: 0.8371 - val_loss: 0.9905 - val_categorical_accuracy: 0.7000\n",
            "Epoch 15/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.4591 - categorical_accuracy: 0.8401\n",
            "Epoch 00015: saving model to model_init_2021-06-2718_19_46.453166/model-00015-0.459-0.840-0.722-0.720.h5\n",
            "21/21 [==============================] - 69s 3s/step - loss: 0.4591 - categorical_accuracy: 0.8401 - val_loss: 0.7219 - val_categorical_accuracy: 0.7200\n",
            "Epoch 16/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.3759 - categorical_accuracy: 0.8643\n",
            "Epoch 00016: saving model to model_init_2021-06-2718_19_46.453166/model-00016-0.376-0.864-0.807-0.690.h5\n",
            "21/21 [==============================] - 68s 3s/step - loss: 0.3759 - categorical_accuracy: 0.8643 - val_loss: 0.8075 - val_categorical_accuracy: 0.6900\n",
            "Epoch 17/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.4183 - categorical_accuracy: 0.8703\n",
            "Epoch 00017: saving model to model_init_2021-06-2718_19_46.453166/model-00017-0.418-0.870-0.732-0.750.h5\n",
            "21/21 [==============================] - 70s 3s/step - loss: 0.4183 - categorical_accuracy: 0.8703 - val_loss: 0.7321 - val_categorical_accuracy: 0.7500\n",
            "Epoch 18/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.3411 - categorical_accuracy: 0.8944\n",
            "Epoch 00018: saving model to model_init_2021-06-2718_19_46.453166/model-00018-0.341-0.894-1.364-0.640.h5\n",
            "\n",
            "Epoch 00018: ReduceLROnPlateau reducing learning rate to 0.00020000000949949026.\n",
            "21/21 [==============================] - 70s 3s/step - loss: 0.3411 - categorical_accuracy: 0.8944 - val_loss: 1.3640 - val_categorical_accuracy: 0.6400\n",
            "Epoch 19/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.3288 - categorical_accuracy: 0.8854\n",
            "Epoch 00019: saving model to model_init_2021-06-2718_19_46.453166/model-00019-0.329-0.885-0.680-0.740.h5\n",
            "21/21 [==============================] - 70s 3s/step - loss: 0.3288 - categorical_accuracy: 0.8854 - val_loss: 0.6797 - val_categorical_accuracy: 0.7400\n",
            "Epoch 20/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.3015 - categorical_accuracy: 0.9095\n",
            "Epoch 00020: saving model to model_init_2021-06-2718_19_46.453166/model-00020-0.302-0.910-0.705-0.780.h5\n",
            "21/21 [==============================] - 67s 3s/step - loss: 0.3015 - categorical_accuracy: 0.9095 - val_loss: 0.7048 - val_categorical_accuracy: 0.7800\n",
            "Epoch 21/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.2653 - categorical_accuracy: 0.9050\n",
            "Epoch 00021: saving model to model_init_2021-06-2718_19_46.453166/model-00021-0.265-0.905-0.735-0.790.h5\n",
            "21/21 [==============================] - 70s 3s/step - loss: 0.2653 - categorical_accuracy: 0.9050 - val_loss: 0.7350 - val_categorical_accuracy: 0.7900\n",
            "Epoch 22/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.2323 - categorical_accuracy: 0.9186\n",
            "Epoch 00022: saving model to model_init_2021-06-2718_19_46.453166/model-00022-0.232-0.919-0.796-0.730.h5\n",
            "\n",
            "Epoch 00022: ReduceLROnPlateau reducing learning rate to 4.0000001899898055e-05.\n",
            "21/21 [==============================] - 71s 3s/step - loss: 0.2323 - categorical_accuracy: 0.9186 - val_loss: 0.7959 - val_categorical_accuracy: 0.7300\n",
            "Epoch 23/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.2673 - categorical_accuracy: 0.9110\n",
            "Epoch 00023: saving model to model_init_2021-06-2718_19_46.453166/model-00023-0.267-0.911-0.725-0.760.h5\n",
            "21/21 [==============================] - 70s 3s/step - loss: 0.2673 - categorical_accuracy: 0.9110 - val_loss: 0.7250 - val_categorical_accuracy: 0.7600\n",
            "Epoch 24/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.2497 - categorical_accuracy: 0.9246\n",
            "Epoch 00024: saving model to model_init_2021-06-2718_19_46.453166/model-00024-0.250-0.925-0.680-0.820.h5\n",
            "21/21 [==============================] - 67s 3s/step - loss: 0.2497 - categorical_accuracy: 0.9246 - val_loss: 0.6799 - val_categorical_accuracy: 0.8200\n",
            "Epoch 25/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.2490 - categorical_accuracy: 0.9336\n",
            "Epoch 00025: saving model to model_init_2021-06-2718_19_46.453166/model-00025-0.249-0.934-0.722-0.790.h5\n",
            "\n",
            "Epoch 00025: ReduceLROnPlateau reducing learning rate to 8.000000525498762e-06.\n",
            "21/21 [==============================] - 70s 3s/step - loss: 0.2490 - categorical_accuracy: 0.9336 - val_loss: 0.7221 - val_categorical_accuracy: 0.7900\n",
            "Epoch 26/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.2519 - categorical_accuracy: 0.9125\n",
            "Epoch 00026: saving model to model_init_2021-06-2718_19_46.453166/model-00026-0.252-0.913-0.587-0.820.h5\n",
            "21/21 [==============================] - 70s 3s/step - loss: 0.2519 - categorical_accuracy: 0.9125 - val_loss: 0.5865 - val_categorical_accuracy: 0.8200\n",
            "Epoch 27/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.2376 - categorical_accuracy: 0.9216\n",
            "Epoch 00027: saving model to model_init_2021-06-2718_19_46.453166/model-00027-0.238-0.922-0.822-0.740.h5\n",
            "21/21 [==============================] - 70s 3s/step - loss: 0.2376 - categorical_accuracy: 0.9216 - val_loss: 0.8215 - val_categorical_accuracy: 0.7400\n",
            "Epoch 28/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.2246 - categorical_accuracy: 0.9291\n",
            "Epoch 00028: saving model to model_init_2021-06-2718_19_46.453166/model-00028-0.225-0.929-0.721-0.770.h5\n",
            "21/21 [==============================] - 69s 3s/step - loss: 0.2246 - categorical_accuracy: 0.9291 - val_loss: 0.7212 - val_categorical_accuracy: 0.7700\n",
            "Epoch 29/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.2278 - categorical_accuracy: 0.9336\n",
            "Epoch 00029: saving model to model_init_2021-06-2718_19_46.453166/model-00029-0.228-0.934-0.710-0.780.h5\n",
            "\n",
            "Epoch 00029: ReduceLROnPlateau reducing learning rate to 1.6000001778593287e-06.\n",
            "21/21 [==============================] - 72s 3s/step - loss: 0.2278 - categorical_accuracy: 0.9336 - val_loss: 0.7095 - val_categorical_accuracy: 0.7800\n",
            "Epoch 30/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.2265 - categorical_accuracy: 0.9306\n",
            "Epoch 00030: saving model to model_init_2021-06-2718_19_46.453166/model-00030-0.227-0.931-0.762-0.740.h5\n",
            "21/21 [==============================] - 71s 3s/step - loss: 0.2265 - categorical_accuracy: 0.9306 - val_loss: 0.7619 - val_categorical_accuracy: 0.7400\n",
            "Epoch 31/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.2438 - categorical_accuracy: 0.9231\n",
            "Epoch 00031: saving model to model_init_2021-06-2718_19_46.453166/model-00031-0.244-0.923-0.540-0.820.h5\n",
            "21/21 [==============================] - 72s 3s/step - loss: 0.2438 - categorical_accuracy: 0.9231 - val_loss: 0.5401 - val_categorical_accuracy: 0.8200\n",
            "Epoch 32/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.2443 - categorical_accuracy: 0.9170\n",
            "Epoch 00032: saving model to model_init_2021-06-2718_19_46.453166/model-00032-0.244-0.917-0.658-0.800.h5\n",
            "21/21 [==============================] - 69s 3s/step - loss: 0.2443 - categorical_accuracy: 0.9170 - val_loss: 0.6581 - val_categorical_accuracy: 0.8000\n",
            "Epoch 33/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.2364 - categorical_accuracy: 0.9291\n",
            "Epoch 00033: saving model to model_init_2021-06-2718_19_46.453166/model-00033-0.236-0.929-0.707-0.780.h5\n",
            "21/21 [==============================] - 72s 3s/step - loss: 0.2364 - categorical_accuracy: 0.9291 - val_loss: 0.7067 - val_categorical_accuracy: 0.7800\n",
            "Epoch 34/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.2017 - categorical_accuracy: 0.9442\n",
            "Epoch 00034: saving model to model_init_2021-06-2718_19_46.453166/model-00034-0.202-0.944-0.597-0.810.h5\n",
            "\n",
            "Epoch 00034: ReduceLROnPlateau reducing learning rate to 3.200000264769187e-07.\n",
            "21/21 [==============================] - 71s 3s/step - loss: 0.2017 - categorical_accuracy: 0.9442 - val_loss: 0.5969 - val_categorical_accuracy: 0.8100\n",
            "Epoch 35/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.2273 - categorical_accuracy: 0.9155\n",
            "Epoch 00035: saving model to model_init_2021-06-2718_19_46.453166/model-00035-0.227-0.916-0.729-0.780.h5\n",
            "21/21 [==============================] - 72s 3s/step - loss: 0.2273 - categorical_accuracy: 0.9155 - val_loss: 0.7292 - val_categorical_accuracy: 0.7800\n",
            "Epoch 36/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.2278 - categorical_accuracy: 0.9170\n",
            "Epoch 00036: saving model to model_init_2021-06-2718_19_46.453166/model-00036-0.228-0.917-0.699-0.780.h5\n",
            "21/21 [==============================] - 68s 3s/step - loss: 0.2278 - categorical_accuracy: 0.9170 - val_loss: 0.6986 - val_categorical_accuracy: 0.7800\n",
            "Epoch 37/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.2586 - categorical_accuracy: 0.9140\n",
            "Epoch 00037: saving model to model_init_2021-06-2718_19_46.453166/model-00037-0.259-0.914-0.707-0.780.h5\n",
            "\n",
            "Epoch 00037: ReduceLROnPlateau reducing learning rate to 6.400000529538374e-08.\n",
            "21/21 [==============================] - 72s 3s/step - loss: 0.2586 - categorical_accuracy: 0.9140 - val_loss: 0.7067 - val_categorical_accuracy: 0.7800\n",
            "Epoch 38/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.2179 - categorical_accuracy: 0.9336\n",
            "Epoch 00038: saving model to model_init_2021-06-2718_19_46.453166/model-00038-0.218-0.934-0.749-0.740.h5\n",
            "21/21 [==============================] - 72s 3s/step - loss: 0.2179 - categorical_accuracy: 0.9336 - val_loss: 0.7493 - val_categorical_accuracy: 0.7400\n",
            "Epoch 39/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.2338 - categorical_accuracy: 0.9216\n",
            "Epoch 00039: saving model to model_init_2021-06-2718_19_46.453166/model-00039-0.234-0.922-0.638-0.790.h5\n",
            "21/21 [==============================] - 72s 3s/step - loss: 0.2338 - categorical_accuracy: 0.9216 - val_loss: 0.6381 - val_categorical_accuracy: 0.7900\n",
            "Epoch 40/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.2890 - categorical_accuracy: 0.9110\n",
            "Epoch 00040: saving model to model_init_2021-06-2718_19_46.453166/model-00040-0.289-0.911-0.704-0.780.h5\n",
            "\n",
            "Epoch 00040: ReduceLROnPlateau reducing learning rate to 1.2800001059076749e-08.\n",
            "21/21 [==============================] - 69s 3s/step - loss: 0.2890 - categorical_accuracy: 0.9110 - val_loss: 0.7044 - val_categorical_accuracy: 0.7800\n",
            "Epoch 41/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.2826 - categorical_accuracy: 0.9140\n",
            "Epoch 00041: saving model to model_init_2021-06-2718_19_46.453166/model-00041-0.283-0.914-0.705-0.780.h5\n",
            "21/21 [==============================] - 72s 3s/step - loss: 0.2826 - categorical_accuracy: 0.9140 - val_loss: 0.7054 - val_categorical_accuracy: 0.7800\n",
            "Epoch 42/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.2361 - categorical_accuracy: 0.9351\n",
            "Epoch 00042: saving model to model_init_2021-06-2718_19_46.453166/model-00042-0.236-0.935-0.768-0.740.h5\n",
            "21/21 [==============================] - 72s 3s/step - loss: 0.2361 - categorical_accuracy: 0.9351 - val_loss: 0.7683 - val_categorical_accuracy: 0.7400\n",
            "Epoch 43/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.2560 - categorical_accuracy: 0.9246\n",
            "Epoch 00043: saving model to model_init_2021-06-2718_19_46.453166/model-00043-0.256-0.925-0.709-0.800.h5\n",
            "\n",
            "Epoch 00043: ReduceLROnPlateau reducing learning rate to 2.5600002118153498e-09.\n",
            "21/21 [==============================] - 72s 3s/step - loss: 0.2560 - categorical_accuracy: 0.9246 - val_loss: 0.7087 - val_categorical_accuracy: 0.8000\n",
            "Epoch 44/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.2472 - categorical_accuracy: 0.9201\n",
            "Epoch 00044: saving model to model_init_2021-06-2718_19_46.453166/model-00044-0.247-0.920-0.681-0.800.h5\n",
            "21/21 [==============================] - 69s 3s/step - loss: 0.2472 - categorical_accuracy: 0.9201 - val_loss: 0.6806 - val_categorical_accuracy: 0.8000\n",
            "Epoch 45/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.2345 - categorical_accuracy: 0.9231\n",
            "Epoch 00045: saving model to model_init_2021-06-2718_19_46.453166/model-00045-0.235-0.923-0.706-0.780.h5\n",
            "21/21 [==============================] - 72s 3s/step - loss: 0.2345 - categorical_accuracy: 0.9231 - val_loss: 0.7064 - val_categorical_accuracy: 0.7800\n",
            "Epoch 46/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.2518 - categorical_accuracy: 0.9186\n",
            "Epoch 00046: saving model to model_init_2021-06-2718_19_46.453166/model-00046-0.252-0.919-0.597-0.810.h5\n",
            "\n",
            "Epoch 00046: ReduceLROnPlateau reducing learning rate to 5.1200004236307e-10.\n",
            "21/21 [==============================] - 71s 3s/step - loss: 0.2518 - categorical_accuracy: 0.9186 - val_loss: 0.5967 - val_categorical_accuracy: 0.8100\n",
            "Epoch 47/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.2758 - categorical_accuracy: 0.9065\n",
            "Epoch 00047: saving model to model_init_2021-06-2718_19_46.453166/model-00047-0.276-0.906-0.784-0.750.h5\n",
            "21/21 [==============================] - 71s 3s/step - loss: 0.2758 - categorical_accuracy: 0.9065 - val_loss: 0.7844 - val_categorical_accuracy: 0.7500\n",
            "Epoch 48/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.2351 - categorical_accuracy: 0.9155\n",
            "Epoch 00048: saving model to model_init_2021-06-2718_19_46.453166/model-00048-0.235-0.916-0.686-0.790.h5\n",
            "21/21 [==============================] - 69s 3s/step - loss: 0.2351 - categorical_accuracy: 0.9155 - val_loss: 0.6857 - val_categorical_accuracy: 0.7900\n",
            "Epoch 49/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.2517 - categorical_accuracy: 0.9186\n",
            "Epoch 00049: saving model to model_init_2021-06-2718_19_46.453166/model-00049-0.252-0.919-0.705-0.780.h5\n",
            "\n",
            "Epoch 00049: ReduceLROnPlateau reducing learning rate to 1.0240001069306004e-10.\n",
            "21/21 [==============================] - 71s 3s/step - loss: 0.2517 - categorical_accuracy: 0.9186 - val_loss: 0.7054 - val_categorical_accuracy: 0.7800\n",
            "Epoch 50/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.3005 - categorical_accuracy: 0.9035\n",
            "Epoch 00050: saving model to model_init_2021-06-2718_19_46.453166/model-00050-0.301-0.903-0.750-0.740.h5\n",
            "21/21 [==============================] - 71s 3s/step - loss: 0.3005 - categorical_accuracy: 0.9035 - val_loss: 0.7498 - val_categorical_accuracy: 0.7400\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<tensorflow.python.keras.callbacks.History at 0x7f271c6ca748>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 13
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "4PqepBJHlAfe"
      },
      "source": [
        "### CNN_RNN with Transfer learning (Resnet50)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "tVmPjZKP0gSr"
      },
      "source": [
        "###  GENERATOR to use same preprocessing used in resnet50\n",
        "import tensorflow.keras as K\n",
        "import tensorflow\n",
        "\n",
        "def generator(source_path, folder_list, batch_size):\n",
        "    print( 'Source path = ', source_path, '; batch size =', batch_size)\n",
        "    global frames,x,y,z\n",
        "    frames = 30\n",
        "    img_idx = [i for i in range(0,frames) if i%2==0]#create a list of image numbers you want to use for a particular video\n",
        "    while True:\n",
        "        t = np.random.permutation(folder_list)\n",
        "        num_batches = len(t)//batch_size# calculate the number of batches\n",
        "        x = len(img_idx)\n",
        "        y,z = 224, 224 ####\n",
        "        for batch in range(num_batches): # we iterate over the number of batches\n",
        "            batch_data = np.zeros((batch_size,x,y,z,3)) # x is the number of images you use for each video, (y,z) is the final size of the input images and 3 is the number of channels RGB\n",
        "            batch_labels = np.zeros((batch_size,5)) # batch_labels is the one hot representation of the output\n",
        "            for folder in range(batch_size): # iterate over the batch_size\n",
        "                imgs = os.listdir(source_path+'/'+ t[folder + (batch*batch_size)].split(';')[0]) # read all the images in the folder\n",
        "                for idx,item in enumerate(img_idx): #  Iterate iver the frames/images of a folder to read them in\n",
        "                    image = imread(source_path+'/'+ t[folder + (batch*batch_size)].strip().split(';')[0]+'/'+imgs[item]).astype(np.float32)\n",
        "#                     print(image.shape)\n",
        "#                     plt.imshow(image/256)\n",
        "#                     return\n",
        "\n",
        "                    if image.shape[0]!=y or image.shape[1]!=z:\n",
        "                        image = tf.image.resize(image,(y,z))\n",
        "#                         image = tensorflow.keras.applications.resnet50.preprocess_input(image)\n",
        "#                         image = resize(image,[y,z], anti_aliasing=True)  --- Added layer for this\n",
        "                    #crop the images and resize them. Note that the images are of 2 different shape \n",
        "                    #and the conv3D will throw error if the inputs in a batch have different shapes\n",
        "                    \n",
        "                    batch_data[folder,idx,:,:,0] = image[:,:,0]\n",
        "                    batch_data[folder,idx,:,:,1] = image[:,:,1]\n",
        "                    batch_data[folder,idx,:,:,2] = image[:,:,2]\n",
        "                    \n",
        "                batch_labels[folder, int(t[folder + (batch*batch_size)].strip().split(';')[2])] = 1\n",
        "            yield batch_data, batch_labels #you yield the batch_data and the batch_labels, remember what does yield do\n",
        "\n",
        "        \n",
        "        # write the code for the remaining data points which are left after full batches\n",
        "        remaining_folders = len(t)%batch_size\n",
        "        if remaining_folders>0:\n",
        "            batch_data = np.zeros((remaining_folders,x,y,z,3)) \n",
        "            batch_labels = np.zeros((remaining_folders,5)) \n",
        "            for folder in range(remaining_folders): \n",
        "                imgs = os.listdir(source_path+'/'+ t[folder + (num_batches*batch_size)].split(';')[0]) # read all the images in the folder\n",
        "                for idx,item in enumerate(img_idx): #  Iterate iver the frames/images of a folder to read them in\n",
        "                    image = imread(source_path+'/'+ t[folder + (num_batches*batch_size)].strip().split(';')[0]+'/'+imgs[item]).astype(np.float32)\n",
        "                    \n",
        "                    if image.shape[0]!=y or image.shape[1]!=z:\n",
        "                        image = tensorflow.keras.applications.resnet50.preprocess_input(image)\n",
        "                        image = tf.image.resize(image,(y,z))\n",
        "#                         image = resize(image,[y,z], anti_aliasing=True)\n",
        "                    #crop the images and resize them. Note that the images are of 2 different shape \n",
        "                    #and the conv3D will throw error if the inputs in a batch have different shapes\n",
        "                    \n",
        "                    batch_data[folder,idx,:,:,0] = image[:,:,0]\n",
        "                    batch_data[folder,idx,:,:,1] = image[:,:,1]\n",
        "                    batch_data[folder,idx,:,:,2] = image[:,:,2]\n",
        "                    \n",
        "                batch_labels[folder, int(t[folder + (num_batches*batch_size)].strip().split(';')[2])] = 1\n",
        "            yield batch_data, batch_labels \n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NZ53tsX90gSs",
        "outputId": "df3db86a-17ab-48ff-cfd7-1d23e6ee1032"
      },
      "source": [
        "curr_dt_time = datetime.datetime.now()\n",
        "train_path = '/home/datasets/Project_data/train'\n",
        "val_path = '/home/datasets/Project_data/val'\n",
        "num_train_sequences = len(train_doc)\n",
        "print('# training sequences =', num_train_sequences)\n",
        "num_val_sequences = len(val_doc)\n",
        "print('# validation sequences =', num_val_sequences)\n",
        "num_epochs = 100 # choose the number of epochs\n",
        "print ('# epochs =', num_epochs)\n",
        "\n",
        "a = generator(train_path, train_doc, batch_size)\n",
        "item = next(a)\n",
        "print(type(item),item[0].shape)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "# training sequences = 663\n",
            "# validation sequences = 100\n",
            "# epochs = 50\n",
            "Source path =  /home/datasets/Project_data/train ; batch size = 64\n",
            "<class 'tuple'> (64, 15, 224, 224, 3)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dmi9PgCD0gSt"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "kOt38OAjlAff",
        "outputId": "17d06598-d971-4688-c4e9-b852bc5811f3"
      },
      "source": [
        "####\n",
        "# CNN-RNN with Transfer learning, Accuracy 64/90\n",
        "####\n",
        "\n",
        "from tensorflow.keras.models import Sequential, Model\n",
        "from tensorflow.keras.layers import GlobalAveragePooling2D, Dense,TimeDistributed, LSTM, GRU, Dropout, Flatten, BatchNormalization, Activation, Conv2D,Conv3D, MaxPooling2D, MaxPooling3D\n",
        "from tensorflow.keras.callbacks import ModelCheckpoint, ReduceLROnPlateau\n",
        "from tensorflow.keras.regularizers import L2 #vs l2\n",
        "from tensorflow.keras import optimizers\n",
        "from tensorflow.keras.applications import ResNet50\n",
        "from tensorflow.keras.applications.resnet50 import preprocess_input\n",
        "\n",
        "#write your model here\n",
        "model = Sequential()\n",
        "# model.add(Lambda(lambda image: tensorflow.image.resize(image, (y,z))))\n",
        "\n",
        "base_model = ResNet50(weights='imagenet', include_top=False)\n",
        "# base_model.summary()    \n",
        "# base_model.layers.pop() ## removing 4x4x2048 layers\n",
        "# base_model = Sequential(base_model.layers[:-2])\n",
        "\n",
        "base_model = Model(base_model.input, base_model.layers[-11].output) ########\n",
        "\n",
        "\n",
        "# As we are using ResNet model only for feature extraction and not adjusting the weights\n",
        "# we freeze the layers in base model\n",
        "for layer in base_model.layers[:143]:\n",
        "    layer.trainable = False\n",
        "\n",
        "for layer in base_model.layers:\n",
        "    print(layer.name,layer.output_shape,layer.trainable)\n",
        "\n",
        "# base_model.summary()\n",
        "# Adding our own layer \n",
        "model.add(TimeDistributed(base_model))\n",
        "\n",
        "# model.add(TimeDistributed(Conv2D(128, (3, 3) , padding = \"SAME\",\n",
        "#                         input_shape=(y, z, 3)))) #### Large batch_size might not fit params into memory\n",
        "# model.add(TimeDistributed(Activation('relu')))\n",
        "# model.add(TimeDistributed(Conv2D(128, ( 3, 3), padding = \"SAME\")))\n",
        "# model.add(TimeDistributed(Activation('relu')))\n",
        "# model.add(TimeDistributed(MaxPooling2D(pool_size=(2, 2))))\n",
        "# model.add(TimeDistributed(Dropout(0.5)))\n",
        "\n",
        "# model.add(TimeDistributed(Conv2D(64, (3, 3) , padding = \"SAME\"))) \n",
        "# model.add(TimeDistributed(Activation('relu')))\n",
        "# model.add(TimeDistributed(Conv2D(64, ( 3, 3), padding = \"SAME\")))\n",
        "# model.add(TimeDistributed(Activation('relu')))\n",
        "# model.add(TimeDistributed(MaxPooling2D(pool_size=(2, 2))))\n",
        "# model.add(TimeDistributed(Dropout(0.5)))\n",
        "\n",
        "## reduce filters from 256 -> 32\n",
        "# model.add(TimeDistributed(Conv2D(32, (3, 3) , padding = \"SAME\"))) \n",
        "# model.add(TimeDistributed(Activation('relu')))\n",
        "# model.add(TimeDistributed(Conv2D(32, ( 3, 3), padding = \"SAME\")))\n",
        "# model.add(TimeDistributed(Activation('relu')))\n",
        "# model.add(TimeDistributed(MaxPooling2D(pool_size=(2, 2))))\n",
        "# model.add(TimeDistributed(Dropout(0.5)))\n",
        "\n",
        "# model.add(pretrained)\n",
        "model.add(TimeDistributed(Flatten()))\n",
        "model.add(LSTM(32\n",
        "#                input_shape = [None,x]\n",
        "#                 activation=\"relu\",\n",
        "#                 dropout = 0.5\n",
        "              ))    \n",
        "model.add(Dropout(0.5))\n",
        "\n",
        "## number of params - 74240 = 4 set of wts * ( 128 output  * ( 16 emb inputdim + 1 bias) +   128 output^2)\n",
        "\n",
        "# Layer 5 - Dense FFN with Dropout\n",
        "model.add(BatchNormalization())\n",
        "model.add(Dense(128))\n",
        "model.add(Dropout(0.5))\n",
        "model.add(BatchNormalization())   ####### better if removed\n",
        "model.add(Dense(5, \n",
        "#                 kernel_initializer = weights,\n",
        "                    name='output'))\n",
        "model.add(Activation('softmax'))\n",
        "\n"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "input_14 [(None, None, None, 3)] False\n",
            "conv1_pad (None, None, None, 3) False\n",
            "conv1_conv (None, None, None, 64) False\n",
            "conv1_bn (None, None, None, 64) False\n",
            "conv1_relu (None, None, None, 64) False\n",
            "pool1_pad (None, None, None, 64) False\n",
            "pool1_pool (None, None, None, 64) False\n",
            "conv2_block1_1_conv (None, None, None, 64) False\n",
            "conv2_block1_1_bn (None, None, None, 64) False\n",
            "conv2_block1_1_relu (None, None, None, 64) False\n",
            "conv2_block1_2_conv (None, None, None, 64) False\n",
            "conv2_block1_2_bn (None, None, None, 64) False\n",
            "conv2_block1_2_relu (None, None, None, 64) False\n",
            "conv2_block1_0_conv (None, None, None, 256) False\n",
            "conv2_block1_3_conv (None, None, None, 256) False\n",
            "conv2_block1_0_bn (None, None, None, 256) False\n",
            "conv2_block1_3_bn (None, None, None, 256) False\n",
            "conv2_block1_add (None, None, None, 256) False\n",
            "conv2_block1_out (None, None, None, 256) False\n",
            "conv2_block2_1_conv (None, None, None, 64) False\n",
            "conv2_block2_1_bn (None, None, None, 64) False\n",
            "conv2_block2_1_relu (None, None, None, 64) False\n",
            "conv2_block2_2_conv (None, None, None, 64) False\n",
            "conv2_block2_2_bn (None, None, None, 64) False\n",
            "conv2_block2_2_relu (None, None, None, 64) False\n",
            "conv2_block2_3_conv (None, None, None, 256) False\n",
            "conv2_block2_3_bn (None, None, None, 256) False\n",
            "conv2_block2_add (None, None, None, 256) False\n",
            "conv2_block2_out (None, None, None, 256) False\n",
            "conv2_block3_1_conv (None, None, None, 64) False\n",
            "conv2_block3_1_bn (None, None, None, 64) False\n",
            "conv2_block3_1_relu (None, None, None, 64) False\n",
            "conv2_block3_2_conv (None, None, None, 64) False\n",
            "conv2_block3_2_bn (None, None, None, 64) False\n",
            "conv2_block3_2_relu (None, None, None, 64) False\n",
            "conv2_block3_3_conv (None, None, None, 256) False\n",
            "conv2_block3_3_bn (None, None, None, 256) False\n",
            "conv2_block3_add (None, None, None, 256) False\n",
            "conv2_block3_out (None, None, None, 256) False\n",
            "conv3_block1_1_conv (None, None, None, 128) False\n",
            "conv3_block1_1_bn (None, None, None, 128) False\n",
            "conv3_block1_1_relu (None, None, None, 128) False\n",
            "conv3_block1_2_conv (None, None, None, 128) False\n",
            "conv3_block1_2_bn (None, None, None, 128) False\n",
            "conv3_block1_2_relu (None, None, None, 128) False\n",
            "conv3_block1_0_conv (None, None, None, 512) False\n",
            "conv3_block1_3_conv (None, None, None, 512) False\n",
            "conv3_block1_0_bn (None, None, None, 512) False\n",
            "conv3_block1_3_bn (None, None, None, 512) False\n",
            "conv3_block1_add (None, None, None, 512) False\n",
            "conv3_block1_out (None, None, None, 512) False\n",
            "conv3_block2_1_conv (None, None, None, 128) False\n",
            "conv3_block2_1_bn (None, None, None, 128) False\n",
            "conv3_block2_1_relu (None, None, None, 128) False\n",
            "conv3_block2_2_conv (None, None, None, 128) False\n",
            "conv3_block2_2_bn (None, None, None, 128) False\n",
            "conv3_block2_2_relu (None, None, None, 128) False\n",
            "conv3_block2_3_conv (None, None, None, 512) False\n",
            "conv3_block2_3_bn (None, None, None, 512) False\n",
            "conv3_block2_add (None, None, None, 512) False\n",
            "conv3_block2_out (None, None, None, 512) False\n",
            "conv3_block3_1_conv (None, None, None, 128) False\n",
            "conv3_block3_1_bn (None, None, None, 128) False\n",
            "conv3_block3_1_relu (None, None, None, 128) False\n",
            "conv3_block3_2_conv (None, None, None, 128) False\n",
            "conv3_block3_2_bn (None, None, None, 128) False\n",
            "conv3_block3_2_relu (None, None, None, 128) False\n",
            "conv3_block3_3_conv (None, None, None, 512) False\n",
            "conv3_block3_3_bn (None, None, None, 512) False\n",
            "conv3_block3_add (None, None, None, 512) False\n",
            "conv3_block3_out (None, None, None, 512) False\n",
            "conv3_block4_1_conv (None, None, None, 128) False\n",
            "conv3_block4_1_bn (None, None, None, 128) False\n",
            "conv3_block4_1_relu (None, None, None, 128) False\n",
            "conv3_block4_2_conv (None, None, None, 128) False\n",
            "conv3_block4_2_bn (None, None, None, 128) False\n",
            "conv3_block4_2_relu (None, None, None, 128) False\n",
            "conv3_block4_3_conv (None, None, None, 512) False\n",
            "conv3_block4_3_bn (None, None, None, 512) False\n",
            "conv3_block4_add (None, None, None, 512) False\n",
            "conv3_block4_out (None, None, None, 512) False\n",
            "conv4_block1_1_conv (None, None, None, 256) False\n",
            "conv4_block1_1_bn (None, None, None, 256) False\n",
            "conv4_block1_1_relu (None, None, None, 256) False\n",
            "conv4_block1_2_conv (None, None, None, 256) False\n",
            "conv4_block1_2_bn (None, None, None, 256) False\n",
            "conv4_block1_2_relu (None, None, None, 256) False\n",
            "conv4_block1_0_conv (None, None, None, 1024) False\n",
            "conv4_block1_3_conv (None, None, None, 1024) False\n",
            "conv4_block1_0_bn (None, None, None, 1024) False\n",
            "conv4_block1_3_bn (None, None, None, 1024) False\n",
            "conv4_block1_add (None, None, None, 1024) False\n",
            "conv4_block1_out (None, None, None, 1024) False\n",
            "conv4_block2_1_conv (None, None, None, 256) False\n",
            "conv4_block2_1_bn (None, None, None, 256) False\n",
            "conv4_block2_1_relu (None, None, None, 256) False\n",
            "conv4_block2_2_conv (None, None, None, 256) False\n",
            "conv4_block2_2_bn (None, None, None, 256) False\n",
            "conv4_block2_2_relu (None, None, None, 256) False\n",
            "conv4_block2_3_conv (None, None, None, 1024) False\n",
            "conv4_block2_3_bn (None, None, None, 1024) False\n",
            "conv4_block2_add (None, None, None, 1024) False\n",
            "conv4_block2_out (None, None, None, 1024) False\n",
            "conv4_block3_1_conv (None, None, None, 256) False\n",
            "conv4_block3_1_bn (None, None, None, 256) False\n",
            "conv4_block3_1_relu (None, None, None, 256) False\n",
            "conv4_block3_2_conv (None, None, None, 256) False\n",
            "conv4_block3_2_bn (None, None, None, 256) False\n",
            "conv4_block3_2_relu (None, None, None, 256) False\n",
            "conv4_block3_3_conv (None, None, None, 1024) False\n",
            "conv4_block3_3_bn (None, None, None, 1024) False\n",
            "conv4_block3_add (None, None, None, 1024) False\n",
            "conv4_block3_out (None, None, None, 1024) False\n",
            "conv4_block4_1_conv (None, None, None, 256) False\n",
            "conv4_block4_1_bn (None, None, None, 256) False\n",
            "conv4_block4_1_relu (None, None, None, 256) False\n",
            "conv4_block4_2_conv (None, None, None, 256) False\n",
            "conv4_block4_2_bn (None, None, None, 256) False\n",
            "conv4_block4_2_relu (None, None, None, 256) False\n",
            "conv4_block4_3_conv (None, None, None, 1024) False\n",
            "conv4_block4_3_bn (None, None, None, 1024) False\n",
            "conv4_block4_add (None, None, None, 1024) False\n",
            "conv4_block4_out (None, None, None, 1024) False\n",
            "conv4_block5_1_conv (None, None, None, 256) False\n",
            "conv4_block5_1_bn (None, None, None, 256) False\n",
            "conv4_block5_1_relu (None, None, None, 256) False\n",
            "conv4_block5_2_conv (None, None, None, 256) False\n",
            "conv4_block5_2_bn (None, None, None, 256) False\n",
            "conv4_block5_2_relu (None, None, None, 256) False\n",
            "conv4_block5_3_conv (None, None, None, 1024) False\n",
            "conv4_block5_3_bn (None, None, None, 1024) False\n",
            "conv4_block5_add (None, None, None, 1024) False\n",
            "conv4_block5_out (None, None, None, 1024) False\n",
            "conv4_block6_1_conv (None, None, None, 256) False\n",
            "conv4_block6_1_bn (None, None, None, 256) False\n",
            "conv4_block6_1_relu (None, None, None, 256) False\n",
            "conv4_block6_2_conv (None, None, None, 256) False\n",
            "conv4_block6_2_bn (None, None, None, 256) False\n",
            "conv4_block6_2_relu (None, None, None, 256) False\n",
            "conv4_block6_3_conv (None, None, None, 1024) False\n",
            "conv4_block6_3_bn (None, None, None, 1024) False\n",
            "conv4_block6_add (None, None, None, 1024) False\n",
            "conv4_block6_out (None, None, None, 1024) False\n",
            "conv5_block1_1_conv (None, None, None, 512) True\n",
            "conv5_block1_1_bn (None, None, None, 512) True\n",
            "conv5_block1_1_relu (None, None, None, 512) True\n",
            "conv5_block1_2_conv (None, None, None, 512) True\n",
            "conv5_block1_2_bn (None, None, None, 512) True\n",
            "conv5_block1_2_relu (None, None, None, 512) True\n",
            "conv5_block1_0_conv (None, None, None, 2048) True\n",
            "conv5_block1_3_conv (None, None, None, 2048) True\n",
            "conv5_block1_0_bn (None, None, None, 2048) True\n",
            "conv5_block1_3_bn (None, None, None, 2048) True\n",
            "conv5_block1_add (None, None, None, 2048) True\n",
            "conv5_block1_out (None, None, None, 2048) True\n",
            "conv5_block2_1_conv (None, None, None, 512) True\n",
            "conv5_block2_1_bn (None, None, None, 512) True\n",
            "conv5_block2_1_relu (None, None, None, 512) True\n",
            "conv5_block2_2_conv (None, None, None, 512) True\n",
            "conv5_block2_2_bn (None, None, None, 512) True\n",
            "conv5_block2_2_relu (None, None, None, 512) True\n",
            "conv5_block2_3_conv (None, None, None, 2048) True\n",
            "conv5_block2_3_bn (None, None, None, 2048) True\n",
            "conv5_block2_add (None, None, None, 2048) True\n",
            "conv5_block2_out (None, None, None, 2048) True\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jev5rXPzlAff",
        "outputId": "f8bc0c0d-9ec1-4822-9c6c-da2748ff4870"
      },
      "source": [
        "learning_rate = 0.001\n",
        "optimiser = optimizers.Adam(lr=learning_rate, clipvalue=1.0)#write your optimizer\n",
        "model.compile(optimizer=optimiser, loss='categorical_crossentropy', metrics=['categorical_accuracy'])\n",
        "\n",
        "model.build(input_shape = [None, x ,y, z ,3]) ##### specify seq\n",
        "\n",
        "print (model.summary())"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"sequential_13\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "time_distributed_50 (TimeDis (None, 15, 7, 7, 2048)    19115904  \n",
            "_________________________________________________________________\n",
            "time_distributed_51 (TimeDis (None, 15, 100352)        0         \n",
            "_________________________________________________________________\n",
            "lstm_9 (LSTM)                (None, 32)                12849280  \n",
            "_________________________________________________________________\n",
            "dropout_26 (Dropout)         (None, 32)                0         \n",
            "_________________________________________________________________\n",
            "batch_normalization_9 (Batch (None, 32)                128       \n",
            "_________________________________________________________________\n",
            "dense_9 (Dense)              (None, 128)               4224      \n",
            "_________________________________________________________________\n",
            "dropout_27 (Dropout)         (None, 128)               0         \n",
            "_________________________________________________________________\n",
            "batch_normalization_10 (Batc (None, 128)               512       \n",
            "_________________________________________________________________\n",
            "output (Dense)               (None, 5)                 645       \n",
            "_________________________________________________________________\n",
            "activation_17 (Activation)   (None, 5)                 0         \n",
            "=================================================================\n",
            "Total params: 31,970,693\n",
            "Trainable params: 23,364,805\n",
            "Non-trainable params: 8,605,888\n",
            "_________________________________________________________________\n",
            "None\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7Y1pZ_FJlAfg"
      },
      "source": [
        "batch_size = 32"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PWYH8Sr0lAfg"
      },
      "source": [
        "train_generator = generator(train_path, train_doc, batch_size)\n",
        "val_generator = generator(val_path, val_doc, batch_size)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "DC6MJ8PIlAfg"
      },
      "source": [
        "model_name = 'model_init' + '_' + str(curr_dt_time).replace(' ','').replace(':','_') + '/'\n",
        "    \n",
        "if not os.path.exists(model_name):\n",
        "    os.mkdir(model_name)\n",
        "        \n",
        "filepath = model_name + 'model-{epoch:05d}-{loss:.3f}-{categorical_accuracy:.3f}-{val_loss:.3f}-{val_categorical_accuracy:.3f}.h5'\n",
        "\n",
        "checkpoint = ModelCheckpoint(filepath, monitor='val_loss', verbose=1, save_best_only=False, save_weights_only=False, mode='auto',save_freq = \"epoch\")\n",
        "\n",
        "LR = ReduceLROnPlateau(monitor='val_loss', factor=0.2, verbose=1, patience=3)# write the REducelronplateau code here\n",
        "callbacks_list = [checkpoint, LR]"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FDjRLVt60gSx",
        "outputId": "b7ffb828-a2a5-491d-cc85-4e174c3b99ae"
      },
      "source": [
        "if (num_train_sequences%batch_size) == 0:\n",
        "    steps_per_epoch = int(num_train_sequences/batch_size)\n",
        "else:\n",
        "    steps_per_epoch = (num_train_sequences//batch_size) + 1\n",
        "\n",
        "if (num_val_sequences%batch_size) == 0:\n",
        "    validation_steps = int(num_val_sequences/batch_size)\n",
        "else:\n",
        "    validation_steps = (num_val_sequences//batch_size) + 1\n",
        "    \n",
        "model.fit(train_generator, steps_per_epoch=steps_per_epoch, epochs=num_epochs, verbose=True, \n",
        "                    callbacks=callbacks_list, validation_data=val_generator, \n",
        "                    validation_steps=validation_steps, class_weight=None, workers=1, initial_epoch=0)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Source path =  /home/datasets/Project_data/train ; batch size = 32\n",
            "Epoch 1/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 2.1369 - categorical_accuracy: 0.2308Source path =  /home/datasets/Project_data/val ; batch size = 32\n",
            "\n",
            "Epoch 00001: saving model to model_init_2021-06-2816_42_42.890086/model-00001-2.137-0.231-1.643-0.160.h5\n",
            "21/21 [==============================] - 107s 5s/step - loss: 2.1369 - categorical_accuracy: 0.2308 - val_loss: 1.6429 - val_categorical_accuracy: 0.1600\n",
            "Epoch 2/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 1.8576 - categorical_accuracy: 0.2534\n",
            "Epoch 00002: saving model to model_init_2021-06-2816_42_42.890086/model-00002-1.858-0.253-1.493-0.330.h5\n",
            "21/21 [==============================] - 111s 5s/step - loss: 1.8576 - categorical_accuracy: 0.2534 - val_loss: 1.4926 - val_categorical_accuracy: 0.3300\n",
            "Epoch 3/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 1.6530 - categorical_accuracy: 0.3198\n",
            "Epoch 00003: saving model to model_init_2021-06-2816_42_42.890086/model-00003-1.653-0.320-1.331-0.420.h5\n",
            "21/21 [==============================] - 110s 5s/step - loss: 1.6530 - categorical_accuracy: 0.3198 - val_loss: 1.3308 - val_categorical_accuracy: 0.4200\n",
            "Epoch 4/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 1.6528 - categorical_accuracy: 0.3213\n",
            "Epoch 00004: saving model to model_init_2021-06-2816_42_42.890086/model-00004-1.653-0.321-1.346-0.350.h5\n",
            "21/21 [==============================] - 96s 5s/step - loss: 1.6528 - categorical_accuracy: 0.3213 - val_loss: 1.3460 - val_categorical_accuracy: 0.3500\n",
            "Epoch 5/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 1.5686 - categorical_accuracy: 0.3137\n",
            "Epoch 00005: saving model to model_init_2021-06-2816_42_42.890086/model-00005-1.569-0.314-1.216-0.400.h5\n",
            "21/21 [==============================] - 100s 5s/step - loss: 1.5686 - categorical_accuracy: 0.3137 - val_loss: 1.2163 - val_categorical_accuracy: 0.4000\n",
            "Epoch 6/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 1.4089 - categorical_accuracy: 0.3922\n",
            "Epoch 00006: saving model to model_init_2021-06-2816_42_42.890086/model-00006-1.409-0.392-1.270-0.410.h5\n",
            "21/21 [==============================] - 100s 5s/step - loss: 1.4089 - categorical_accuracy: 0.3922 - val_loss: 1.2703 - val_categorical_accuracy: 0.4100\n",
            "Epoch 7/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 1.4347 - categorical_accuracy: 0.3695\n",
            "Epoch 00007: saving model to model_init_2021-06-2816_42_42.890086/model-00007-1.435-0.370-1.133-0.480.h5\n",
            "21/21 [==============================] - 101s 5s/step - loss: 1.4347 - categorical_accuracy: 0.3695 - val_loss: 1.1328 - val_categorical_accuracy: 0.4800\n",
            "Epoch 8/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 1.3643 - categorical_accuracy: 0.3876\n",
            "Epoch 00008: saving model to model_init_2021-06-2816_42_42.890086/model-00008-1.364-0.388-1.093-0.530.h5\n",
            "21/21 [==============================] - 100s 5s/step - loss: 1.3643 - categorical_accuracy: 0.3876 - val_loss: 1.0925 - val_categorical_accuracy: 0.5300\n",
            "Epoch 9/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 1.2507 - categorical_accuracy: 0.4374\n",
            "Epoch 00009: saving model to model_init_2021-06-2816_42_42.890086/model-00009-1.251-0.437-0.918-0.700.h5\n",
            "21/21 [==============================] - 102s 5s/step - loss: 1.2507 - categorical_accuracy: 0.4374 - val_loss: 0.9183 - val_categorical_accuracy: 0.7000\n",
            "Epoch 10/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 1.1508 - categorical_accuracy: 0.5143\n",
            "Epoch 00010: saving model to model_init_2021-06-2816_42_42.890086/model-00010-1.151-0.514-0.864-0.710.h5\n",
            "21/21 [==============================] - 103s 5s/step - loss: 1.1508 - categorical_accuracy: 0.5143 - val_loss: 0.8644 - val_categorical_accuracy: 0.7100\n",
            "Epoch 11/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 1.2097 - categorical_accuracy: 0.4977\n",
            "Epoch 00011: saving model to model_init_2021-06-2816_42_42.890086/model-00011-1.210-0.498-1.159-0.540.h5\n",
            "21/21 [==============================] - 103s 5s/step - loss: 1.2097 - categorical_accuracy: 0.4977 - val_loss: 1.1585 - val_categorical_accuracy: 0.5400\n",
            "Epoch 12/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 1.2102 - categorical_accuracy: 0.5038\n",
            "Epoch 00012: saving model to model_init_2021-06-2816_42_42.890086/model-00012-1.210-0.504-0.876-0.660.h5\n",
            "21/21 [==============================] - 98s 5s/step - loss: 1.2102 - categorical_accuracy: 0.5038 - val_loss: 0.8757 - val_categorical_accuracy: 0.6600\n",
            "Epoch 13/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 1.2167 - categorical_accuracy: 0.4887\n",
            "Epoch 00013: saving model to model_init_2021-06-2816_42_42.890086/model-00013-1.217-0.489-0.775-0.700.h5\n",
            "21/21 [==============================] - 99s 5s/step - loss: 1.2167 - categorical_accuracy: 0.4887 - val_loss: 0.7753 - val_categorical_accuracy: 0.7000\n",
            "Epoch 14/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 1.1668 - categorical_accuracy: 0.5264\n",
            "Epoch 00014: saving model to model_init_2021-06-2816_42_42.890086/model-00014-1.167-0.526-0.760-0.740.h5\n",
            "21/21 [==============================] - 99s 5s/step - loss: 1.1668 - categorical_accuracy: 0.5264 - val_loss: 0.7595 - val_categorical_accuracy: 0.7400\n",
            "Epoch 15/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 1.0936 - categorical_accuracy: 0.5400\n",
            "Epoch 00015: saving model to model_init_2021-06-2816_42_42.890086/model-00015-1.094-0.540-0.696-0.730.h5\n",
            "21/21 [==============================] - 100s 5s/step - loss: 1.0936 - categorical_accuracy: 0.5400 - val_loss: 0.6956 - val_categorical_accuracy: 0.7300\n",
            "Epoch 16/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 1.0621 - categorical_accuracy: 0.5581\n",
            "Epoch 00016: saving model to model_init_2021-06-2816_42_42.890086/model-00016-1.062-0.558-0.583-0.800.h5\n",
            "21/21 [==============================] - 97s 5s/step - loss: 1.0621 - categorical_accuracy: 0.5581 - val_loss: 0.5831 - val_categorical_accuracy: 0.8000\n",
            "Epoch 17/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.8893 - categorical_accuracy: 0.5973\n",
            "Epoch 00017: saving model to model_init_2021-06-2816_42_42.890086/model-00017-0.889-0.597-0.520-0.820.h5\n",
            "21/21 [==============================] - 98s 5s/step - loss: 0.8893 - categorical_accuracy: 0.5973 - val_loss: 0.5197 - val_categorical_accuracy: 0.8200\n",
            "Epoch 18/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.9178 - categorical_accuracy: 0.6048\n",
            "Epoch 00018: saving model to model_init_2021-06-2816_42_42.890086/model-00018-0.918-0.605-0.511-0.800.h5\n",
            "21/21 [==============================] - 99s 5s/step - loss: 0.9178 - categorical_accuracy: 0.6048 - val_loss: 0.5113 - val_categorical_accuracy: 0.8000\n",
            "Epoch 19/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.9331 - categorical_accuracy: 0.6094\n",
            "Epoch 00019: saving model to model_init_2021-06-2816_42_42.890086/model-00019-0.933-0.609-0.704-0.800.h5\n",
            "21/21 [==============================] - 98s 5s/step - loss: 0.9331 - categorical_accuracy: 0.6094 - val_loss: 0.7039 - val_categorical_accuracy: 0.8000\n",
            "Epoch 20/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.8784 - categorical_accuracy: 0.6229\n",
            "Epoch 00020: saving model to model_init_2021-06-2816_42_42.890086/model-00020-0.878-0.623-0.560-0.830.h5\n",
            "21/21 [==============================] - 97s 5s/step - loss: 0.8784 - categorical_accuracy: 0.6229 - val_loss: 0.5599 - val_categorical_accuracy: 0.8300\n",
            "Epoch 21/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.9138 - categorical_accuracy: 0.6259\n",
            "Epoch 00021: saving model to model_init_2021-06-2816_42_42.890086/model-00021-0.914-0.626-0.514-0.830.h5\n",
            "\n",
            "Epoch 00021: ReduceLROnPlateau reducing learning rate to 0.00020000000949949026.\n",
            "21/21 [==============================] - 98s 5s/step - loss: 0.9138 - categorical_accuracy: 0.6259 - val_loss: 0.5142 - val_categorical_accuracy: 0.8300\n",
            "Epoch 22/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.8368 - categorical_accuracy: 0.6637\n",
            "Epoch 00022: saving model to model_init_2021-06-2816_42_42.890086/model-00022-0.837-0.664-0.412-0.890.h5\n",
            "21/21 [==============================] - 99s 5s/step - loss: 0.8368 - categorical_accuracy: 0.6637 - val_loss: 0.4121 - val_categorical_accuracy: 0.8900\n",
            "Epoch 23/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.8334 - categorical_accuracy: 0.6410\n",
            "Epoch 00023: saving model to model_init_2021-06-2816_42_42.890086/model-00023-0.833-0.641-0.384-0.900.h5\n",
            "21/21 [==============================] - 99s 5s/step - loss: 0.8334 - categorical_accuracy: 0.6410 - val_loss: 0.3838 - val_categorical_accuracy: 0.9000\n",
            "Epoch 24/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.8565 - categorical_accuracy: 0.6275\n",
            "Epoch 00024: saving model to model_init_2021-06-2816_42_42.890086/model-00024-0.856-0.627-0.397-0.880.h5\n",
            "21/21 [==============================] - 97s 5s/step - loss: 0.8565 - categorical_accuracy: 0.6275 - val_loss: 0.3968 - val_categorical_accuracy: 0.8800\n",
            "Epoch 25/50\n",
            " 9/21 [===========>..................] - ETA: 45s - loss: 0.8062 - categorical_accuracy: 0.6701"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9EvzMGWElAfg",
        "outputId": "22f8bd22-955a-4cd5-cacf-fd43ae9c5ce0"
      },
      "source": [
        "if (num_train_sequences%batch_size) == 0:\n",
        "    steps_per_epoch = int(num_train_sequences/batch_size)\n",
        "else:\n",
        "    steps_per_epoch = (num_train_sequences//batch_size) + 1\n",
        "\n",
        "if (num_val_sequences%batch_size) == 0:\n",
        "    validation_steps = int(num_val_sequences/batch_size)\n",
        "else:\n",
        "    validation_steps = (num_val_sequences//batch_size) + 1\n",
        "    \n",
        "model.fit(train_generator, steps_per_epoch=steps_per_epoch, epochs=num_epochs, verbose=True, \n",
        "                    callbacks=callbacks_list, validation_data=val_generator, \n",
        "                    validation_steps=validation_steps, class_weight=None, workers=1, initial_epoch=0)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Source path =  /home/datasets/Project_data/train ; batch size = 32\n",
            "Epoch 1/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 1.7505 - categorical_accuracy: 0.2383Source path =  /home/datasets/Project_data/val ; batch size = 32\n",
            "\n",
            "Epoch 00001: saving model to model_init_2021-06-2816_42_42.890086/model-00001-1.751-0.238-1.614-0.310.h5\n",
            "21/21 [==============================] - 104s 5s/step - loss: 1.7505 - categorical_accuracy: 0.2383 - val_loss: 1.6144 - val_categorical_accuracy: 0.3100\n",
            "Epoch 2/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 1.6952 - categorical_accuracy: 0.2730\n",
            "Epoch 00002: saving model to model_init_2021-06-2816_42_42.890086/model-00002-1.695-0.273-1.538-0.270.h5\n",
            "21/21 [==============================] - 103s 5s/step - loss: 1.6952 - categorical_accuracy: 0.2730 - val_loss: 1.5378 - val_categorical_accuracy: 0.2700\n",
            "Epoch 3/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 1.6232 - categorical_accuracy: 0.2715\n",
            "Epoch 00003: saving model to model_init_2021-06-2816_42_42.890086/model-00003-1.623-0.271-1.499-0.390.h5\n",
            "21/21 [==============================] - 103s 5s/step - loss: 1.6232 - categorical_accuracy: 0.2715 - val_loss: 1.4990 - val_categorical_accuracy: 0.3900\n",
            "Epoch 4/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 1.5575 - categorical_accuracy: 0.3167\n",
            "Epoch 00004: saving model to model_init_2021-06-2816_42_42.890086/model-00004-1.558-0.317-1.433-0.470.h5\n",
            "21/21 [==============================] - 102s 5s/step - loss: 1.5575 - categorical_accuracy: 0.3167 - val_loss: 1.4331 - val_categorical_accuracy: 0.4700\n",
            "Epoch 5/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 1.4705 - categorical_accuracy: 0.3650\n",
            "Epoch 00005: saving model to model_init_2021-06-2816_42_42.890086/model-00005-1.471-0.365-1.350-0.520.h5\n",
            "21/21 [==============================] - 104s 5s/step - loss: 1.4705 - categorical_accuracy: 0.3650 - val_loss: 1.3497 - val_categorical_accuracy: 0.5200\n",
            "Epoch 6/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 1.4700 - categorical_accuracy: 0.3333\n",
            "Epoch 00006: saving model to model_init_2021-06-2816_42_42.890086/model-00006-1.470-0.333-1.318-0.550.h5\n",
            "21/21 [==============================] - 104s 5s/step - loss: 1.4700 - categorical_accuracy: 0.3333 - val_loss: 1.3175 - val_categorical_accuracy: 0.5500\n",
            "Epoch 7/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 1.3310 - categorical_accuracy: 0.4087\n",
            "Epoch 00007: saving model to model_init_2021-06-2816_42_42.890086/model-00007-1.331-0.409-1.119-0.690.h5\n",
            "21/21 [==============================] - 104s 5s/step - loss: 1.3310 - categorical_accuracy: 0.4087 - val_loss: 1.1186 - val_categorical_accuracy: 0.6900\n",
            "Epoch 8/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 1.1946 - categorical_accuracy: 0.4811\n",
            "Epoch 00008: saving model to model_init_2021-06-2816_42_42.890086/model-00008-1.195-0.481-0.935-0.730.h5\n",
            "21/21 [==============================] - 104s 5s/step - loss: 1.1946 - categorical_accuracy: 0.4811 - val_loss: 0.9354 - val_categorical_accuracy: 0.7300\n",
            "Epoch 9/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 1.0846 - categorical_accuracy: 0.5339\n",
            "Epoch 00009: saving model to model_init_2021-06-2816_42_42.890086/model-00009-1.085-0.534-0.997-0.550.h5\n",
            "21/21 [==============================] - 106s 5s/step - loss: 1.0846 - categorical_accuracy: 0.5339 - val_loss: 0.9966 - val_categorical_accuracy: 0.5500\n",
            "Epoch 10/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 1.0850 - categorical_accuracy: 0.5897\n",
            "Epoch 00010: saving model to model_init_2021-06-2816_42_42.890086/model-00010-1.085-0.590-0.913-0.700.h5\n",
            "21/21 [==============================] - 105s 5s/step - loss: 1.0850 - categorical_accuracy: 0.5897 - val_loss: 0.9126 - val_categorical_accuracy: 0.7000\n",
            "Epoch 11/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.9517 - categorical_accuracy: 0.6139\n",
            "Epoch 00011: saving model to model_init_2021-06-2816_42_42.890086/model-00011-0.952-0.614-0.870-0.770.h5\n",
            "21/21 [==============================] - 104s 5s/step - loss: 0.9517 - categorical_accuracy: 0.6139 - val_loss: 0.8700 - val_categorical_accuracy: 0.7700\n",
            "Epoch 12/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.9436 - categorical_accuracy: 0.6290\n",
            "Epoch 00012: saving model to model_init_2021-06-2816_42_42.890086/model-00012-0.944-0.629-1.006-0.660.h5\n",
            "21/21 [==============================] - 101s 5s/step - loss: 0.9436 - categorical_accuracy: 0.6290 - val_loss: 1.0060 - val_categorical_accuracy: 0.6600\n",
            "Epoch 13/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 1.1173 - categorical_accuracy: 0.5505\n",
            "Epoch 00013: saving model to model_init_2021-06-2816_42_42.890086/model-00013-1.117-0.551-1.086-0.640.h5\n",
            "21/21 [==============================] - 103s 5s/step - loss: 1.1173 - categorical_accuracy: 0.5505 - val_loss: 1.0861 - val_categorical_accuracy: 0.6400\n",
            "Epoch 14/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 1.1300 - categorical_accuracy: 0.5113\n",
            "Epoch 00014: saving model to model_init_2021-06-2816_42_42.890086/model-00014-1.130-0.511-0.767-0.660.h5\n",
            "21/21 [==============================] - 104s 5s/step - loss: 1.1300 - categorical_accuracy: 0.5113 - val_loss: 0.7674 - val_categorical_accuracy: 0.6600\n",
            "Epoch 15/50\n",
            "21/21 [==============================] - ETA: 0s - loss: 0.9939 - categorical_accuracy: 0.5807\n",
            "Epoch 00015: saving model to model_init_2021-06-2816_42_42.890086/model-00015-0.994-0.581-0.968-0.550.h5\n",
            "21/21 [==============================] - 104s 5s/step - loss: 0.9939 - categorical_accuracy: 0.5807 - val_loss: 0.9676 - val_categorical_accuracy: 0.5500\n",
            "Epoch 16/50\n",
            " 9/21 [===========>..................] - ETA: 48s - loss: 0.9259 - categorical_accuracy: 0.6076"
          ],
          "name": "stdout"
        },
        {
          "output_type": "error",
          "ename": "KeyboardInterrupt",
          "evalue": "",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mKeyboardInterrupt\u001b[0m                         Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-43-071c0c33a92e>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m\u001b[0m\n\u001b[1;32m     11\u001b[0m model.fit(train_generator, steps_per_epoch=steps_per_epoch, epochs=num_epochs, verbose=True, \n\u001b[1;32m     12\u001b[0m                     \u001b[0mcallbacks\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mcallbacks_list\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mvalidation_data\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mval_generator\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 13\u001b[0;31m                     validation_steps=validation_steps, class_weight=None, workers=1, initial_epoch=0)\n\u001b[0m",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py\u001b[0m in \u001b[0;36m_method_wrapper\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m    106\u001b[0m   \u001b[0;32mdef\u001b[0m \u001b[0m_method_wrapper\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    107\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_in_multi_worker_mode\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m  \u001b[0;31m# pylint: disable=protected-access\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 108\u001b[0;31m       \u001b[0;32mreturn\u001b[0m \u001b[0mmethod\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    109\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    110\u001b[0m     \u001b[0;31m# Running inside `run_distribute_coordinator` already.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/engine/training.py\u001b[0m in \u001b[0;36mfit\u001b[0;34m(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_batch_size, validation_freq, max_queue_size, workers, use_multiprocessing)\u001b[0m\n\u001b[1;32m   1096\u001b[0m                 batch_size=batch_size):\n\u001b[1;32m   1097\u001b[0m               \u001b[0mcallbacks\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mon_train_batch_begin\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstep\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1098\u001b[0;31m               \u001b[0mtmp_logs\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mtrain_function\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0miterator\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1099\u001b[0m               \u001b[0;32mif\u001b[0m \u001b[0mdata_handler\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mshould_sync\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1100\u001b[0m                 \u001b[0mcontext\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0masync_wait\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/def_function.py\u001b[0m in \u001b[0;36m__call__\u001b[0;34m(self, *args, **kwds)\u001b[0m\n\u001b[1;32m    778\u001b[0m       \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    779\u001b[0m         \u001b[0mcompiler\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m\"nonXla\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 780\u001b[0;31m         \u001b[0mresult\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_call\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwds\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    781\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    782\u001b[0m       \u001b[0mnew_tracing_count\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_get_tracing_count\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/def_function.py\u001b[0m in \u001b[0;36m_call\u001b[0;34m(self, *args, **kwds)\u001b[0m\n\u001b[1;32m    805\u001b[0m       \u001b[0;31m# In this case we have created variables on the first call, so we run the\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    806\u001b[0m       \u001b[0;31m# defunned version which is guaranteed to never create variables.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 807\u001b[0;31m       \u001b[0;32mreturn\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_stateless_fn\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwds\u001b[0m\u001b[0;34m)\u001b[0m  \u001b[0;31m# pylint: disable=not-callable\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    808\u001b[0m     \u001b[0;32melif\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_stateful_fn\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    809\u001b[0m       \u001b[0;31m# Release the lock early so that multiple threads can perform the call\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/function.py\u001b[0m in \u001b[0;36m__call__\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m   2827\u001b[0m     \u001b[0;32mwith\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_lock\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2828\u001b[0m       \u001b[0mgraph_function\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkwargs\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_maybe_define_function\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 2829\u001b[0;31m     \u001b[0;32mreturn\u001b[0m \u001b[0mgraph_function\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_filtered_call\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m  \u001b[0;31m# pylint: disable=protected-access\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   2830\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   2831\u001b[0m   \u001b[0;34m@\u001b[0m\u001b[0mproperty\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/function.py\u001b[0m in \u001b[0;36m_filtered_call\u001b[0;34m(self, args, kwargs, cancellation_manager)\u001b[0m\n\u001b[1;32m   1846\u001b[0m                            resource_variable_ops.BaseResourceVariable))],\n\u001b[1;32m   1847\u001b[0m         \u001b[0mcaptured_inputs\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcaptured_inputs\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1848\u001b[0;31m         cancellation_manager=cancellation_manager)\n\u001b[0m\u001b[1;32m   1849\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1850\u001b[0m   \u001b[0;32mdef\u001b[0m \u001b[0m_call_flat\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcaptured_inputs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mcancellation_manager\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;32mNone\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/function.py\u001b[0m in \u001b[0;36m_call_flat\u001b[0;34m(self, args, captured_inputs, cancellation_manager)\u001b[0m\n\u001b[1;32m   1922\u001b[0m       \u001b[0;31m# No tape is watching; skip to running the function.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1923\u001b[0m       return self._build_call_outputs(self._inference_function.call(\n\u001b[0;32m-> 1924\u001b[0;31m           ctx, args, cancellation_manager=cancellation_manager))\n\u001b[0m\u001b[1;32m   1925\u001b[0m     forward_backward = self._select_forward_and_backward_functions(\n\u001b[1;32m   1926\u001b[0m         \u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/function.py\u001b[0m in \u001b[0;36mcall\u001b[0;34m(self, ctx, args, cancellation_manager)\u001b[0m\n\u001b[1;32m    548\u001b[0m               \u001b[0minputs\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0margs\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    549\u001b[0m               \u001b[0mattrs\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mattrs\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 550\u001b[0;31m               ctx=ctx)\n\u001b[0m\u001b[1;32m    551\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    552\u001b[0m           outputs = execute.execute_with_cancellation(\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/tensorflow/python/eager/execute.py\u001b[0m in \u001b[0;36mquick_execute\u001b[0;34m(op_name, num_outputs, inputs, attrs, ctx, name)\u001b[0m\n\u001b[1;32m     58\u001b[0m     \u001b[0mctx\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mensure_initialized\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     59\u001b[0m     tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,\n\u001b[0;32m---> 60\u001b[0;31m                                         inputs, attrs, num_outputs)\n\u001b[0m\u001b[1;32m     61\u001b[0m   \u001b[0;32mexcept\u001b[0m \u001b[0mcore\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_NotOkStatusException\u001b[0m \u001b[0;32mas\u001b[0m \u001b[0me\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     62\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mname\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mKeyboardInterrupt\u001b[0m: "
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HKmAaef3lAfh"
      },
      "source": [
        "### Val accuracy is more since, drop out is not used at test/validation time , and more neurons are used during testing"
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}