4_video_analytics/01_lets_do_some_video_analysis.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Let's Do Some Video Analysis\n",
    "\n",
    "What better way to learn a concept than to get your hands dirty with code. So let us pick up a sample problem and see how can we do some basic image processing to do some simple analytics on a video. \n",
    "\n",
    "We will implement a method to achieve a quick and dirty solution for a problem but we will discuss various ways through which the problem can be attacked to broaden our understanding of how to build different candidate solutions for a problem and understand/analyse the pros and cons of each of those approaches. \n",
    "\n",
    "Some of the major factors which we will discuss are **FPS Delivered** and **Compute Power** as these are probably the most important factors as far as the deployability of a video analysis algorithm is considered.\n",
    "\n",
    "**Some Ineteresting Reads**:\n",
    "\n",
    "- [Video Analysis using Opencv-Python](https://people.revoledu.com/kardi/tutorial/Python/Video+Analysis+using+OpenCV-Python.html#:~:text=Video%20Analysis%20using%20OpenCV%2DPython&text=This%20tutorial%20is%20a%20practice,numpy%20and%20math%20modules%20installed.)\n",
    "- [Video Analysis - OpenCV Tutorial](https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_video/py_table_of_contents_video/py_table_of_contents_video.html)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Example Problems\n",
    "\n",
    "We have a stock video of a person juggling 3 multicolored balls. The sample problems which we can make from this video are:\n",
    "\n",
    "- Identifying the balls as they juggle\n",
    "- Tracking the balls\n",
    "- Calculate frequency of each ball (1/Time between one juggle)\n",
    "\n",
    "**Video Ref**: https://www.pexels.com/video/person-juggling-balls-854421/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Let us see the video and process it"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 513,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Importing the libraries\n",
    "\n",
    "import cv2\n",
    "import math\n",
    "import time\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 514,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Reading the video file\n",
    "\n",
    "vs = cv2.VideoCapture('..//assets//videos//juggling.mp4')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Control Function\n",
    "\n",
    "We put the visualization code inside a control function so that we can reuse it during experimentation. This function is used to control how we want to process the video. We also add resizing and processing capability by adding _frac_ and _func_ arguments.\n",
    "\n",
    "- _frac_ : fraction to resize video frame. Defaults to 0.2. This is useful for high resolution videos.\n",
    "- _func_ : func to be used to process video. Defaults to None. If not passed then frame is show as it is."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 515,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Defining function to process and visualize given video\n",
    "\n",
    "def processAndShow(vs, func = None, frac = 0.2):\n",
    "    seconds = 0\n",
    "    count = 0\n",
    "    \n",
    "    while vs.isOpened():\n",
    "        ret, fr = vs.read()\n",
    "        \n",
    "        if ret == False:\n",
    "            break\n",
    "        \n",
    "        tic = time.time()\n",
    "        fr = cv2.resize(fr, (int(vs.get(cv2.CAP_PROP_FRAME_WIDTH)*frac), \n",
    "                             int(vs.get(cv2.CAP_PROP_FRAME_HEIGHT)*frac)), fx = frac, fy = frac)\n",
    "        if func is not None:\n",
    "            fr = func(fr)\n",
    "        toc = time.time()\n",
    "        \n",
    "        count+=1\n",
    "        seconds+=toc-tic\n",
    "        \n",
    "        if seconds < 1:\n",
    "            cv2.putText(fr, 'FPS: {0:.2f}'.format(count), (10, 30),  \n",
    "                    cv2.FONT_HERSHEY_SIMPLEX, 1, (255, 255, 255), 1, cv2.LINE_AA)\n",
    "        else:\n",
    "            cv2.putText(fr, 'FPS: {0:.2f}'.format(count/seconds), (10, 30),  \n",
    "                    cv2.FONT_HERSHEY_SIMPLEX, 1, (255, 255, 255), 1, cv2.LINE_AA)\n",
    "            \n",
    "        cv2.imshow('Output Feed', fr)\n",
    "        \n",
    "        if cv2.waitKey(1) & 0xFF == ord('q'):\n",
    "            break\n",
    "    vs.release()\n",
    "    cv2.destroyAllWindows()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 516,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Calling the control function without func for only visualization and no processing\n",
    "\n",
    "# processAndShow(vs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Processing Function\n",
    "\n",
    "We add the analysis code to a processing function and pass it as an argument to our controlling function. This is a simple way to decouple the control and processing logic. \n",
    "\n",
    "- _fr_ : This function should have frame as a **mandatory argument**. The control function should pass each frame to this function.\n",
    "- Any objects to be used in the processing function can be added as default arguments to this function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 517,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Processing function definition\n",
    "\n",
    "def proc(fr, backSub = cv2.createBackgroundSubtractorMOG2()):\n",
    "    \n",
    "    # Step 1. Remove the background information. That is only keep the moving pixels.\n",
    "    fr_ = backSub.apply(fr)\n",
    "    \n",
    "    # Step 2. Apply Morphological Erosion/Dilation with circular/elliptical kernels to refine circular blobs\n",
    "    fr_ = cv2.erode(fr_, cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(5,5)), iterations = 5)\n",
    "    fr_ = cv2.dilate(fr_, cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(7,7)), iterations = 5)\n",
    "        \n",
    "    # Step 3. Extract contours\n",
    "    contours, hierarchy = cv2.findContours(fr_, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)\n",
    "    for c in contours:\n",
    "        contours_poly = cv2.approxPolyDP(c, 3, True)\n",
    "        center, radius = cv2.minEnclosingCircle(contours_poly)\n",
    "        \n",
    "        # Step 4. Filter the contours based on the area, using radius as proxy for area\n",
    "        if radius > 40 and radius < 50:\n",
    "            cv2.circle(fr, (int(center[0]), int(center[1])), int(radius), [255, 0, 0], thickness = 10)\n",
    "\n",
    "    return fr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 518,
   "metadata": {},
   "outputs": [],
   "source": [
    "processAndShow(vs, proc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "py36tf",
   "language": "python",
   "name": "py36tf"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}