03-Demo-ISS-Analysers.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Analysers in Intel® Sytem Studio"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Profiling a Computer Vision Application"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "OpenCV in Intel® Distribution of OpenVINO™ leverages a tracing facility for efficient development of Computer Vision Applications.\n",
    "\n",
    "Tracing framework is built-in directly into the OpenCV source code. To use it, you only need to compile and link your application with modern version of OpenCV.\n",
    "\n",
    "OpenCV tracing relies on the ITT API, an [Intel®-provided profiling framework](https://software.intel.com/en-us/articles/intel-itt-api-open-source)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#The tracing is enabled by default at compile time. \n",
    "#But it's not activated by default at runtime, because it impacts the performance. \n",
    "#It's activated when the environment variable OPENCV_TRACE is set (OPENCV_TRACE=1) \n",
    "#or when OpenCV-based app is running within a specialized profiling software supporting \n",
    "#Instrumentation and Tracing Technology (ITT).\n",
    "\n",
    "%env OPENCV_TRACE=0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using VTune™ Amplifier"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The handiest way to collect a trace is to run your application under control of Intel® VTune™ Amplifier. With VTune™ you can correlate your OpenCV activity with various important metrics for CPU and GPU. Finally, VTune™ captures most Intel APIs beyond OpenCV, for example OpenCL. So with these tools you can see when OpenCV calls are translated into OpenCL tasks.<br><br>\n",
    "![alt text](VTune1.png) <br>\n",
    "Just run analyzed OpenCV Application under Intel® VTune™ Amplifier with enabled \"Analyze user tasks\" option."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!python rm-rf.py runs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#VTune Hotspot analysis\n",
    "!start \"\" cmd /K amplxe-cl \\\n",
    "-collect hotspots \\\n",
    "--result-dir runs/vtune/cpu_run_1 \\\n",
    "--duration 15 \\\n",
    "-- RelWithDebInfo/monitor.exe \\\n",
    "-m=\"pedestrian-and-vehicle-detector-adas-0001.bin\" \\\n",
    "-c=\"pedestrian-and-vehicle-detector-adas-0001.xml\" \\\n",
    "-i=\"car-detection.mp4\" \\\n",
    "-cc=0.7 -e=\"b\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Generate VTune Hotspot analysis report\n",
    "%sx amplxe-cl \\\n",
    "-report hotspots \\\n",
    "-r=runs/vtune/cpu_run_1 \\\n",
    "-- RelWithDebInfo/monitor.exe \\\n",
    "-m=\"pedestrian-and-vehicle-detector-adas-0001.bin\" \\\n",
    "-c=\"pedestrian-and-vehicle-detector-adas-0001.xml\" \\\n",
    "-i=\"car-detection.mp4\" \\\n",
    "-cc=0.7 -e=\"b\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#This example displays a Hotspots report for the r001hs result, \n",
    "#presenting CPU time for the functions of the target in  \n",
    "#descending order starting from the most time-consuming function.\n",
    "\n",
    "%sx amplxe-cl -report hotspots -r runs/vtune/cpu_run_1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%capture\n",
    "#Launch VTune Amplifier 2019 GUI\n",
    "!\"C:\\Program Files (x86)\\IntelSWTools\\VTune Amplifier 2019\\bin64\\amplxe-gui.exe\" runs/vtune/cpu_run_1/cpu_run_1.amplxe"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Using Intel® Advisor™ Amplifier\n",
    "![alt text](Advise1.png) <br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=pink><b>Command Line Interface Reference</b></font>\n",
    "\n",
    "You can use the Intel® Advisor command line interface, advixe-cl, to run analysis and data collection. This makes it possible to automate many tasks or analyze an application running on remote hosts. The results can be viewed through the Intel Advisor GUI, or used to generate various kinds of reports.\n",
    "<br><br>\n",
    "NOTE\n",
    "<br>\n",
    "Before using the command line, you must set command line environment variables.\n",
    "For instructions, see [Setting and Using Intel Advisor Environment Variables](https://software.intel.com/en-us/advisor-user-guide-setting-and-using-intel-advisor-environment-variables)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Survey Report Purpose and Usage"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Run a Survey analysis to generate a <b>Survey Report</b> that offers integrated compiler report data and performance data for your target application all in one place. Optionally run a Trip Counts analysis and/or FLOP analysis to add data to the Survey Report. The Roofline analysis runs a Survey analysis followed by a FLOP analysis automatically."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=blue><b>Survey analysis</b></font> - Identifies:\n",
    "* Where vectorization, or parallelization with threads, will pay off the most\n",
    "* If vectorized loops are providing benefit, and if not, why not\n",
    "* Un-vectorized loops and why they are not vectorized\n",
    "* Performance problems in general"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Survey analysis\n",
    "!start \"\" cmd /C advixe-cl.exe \\\n",
    "--collect survey \\\n",
    "--duration 15 \\\n",
    "--project-dir runs/advise/cpu_run_1 \\\n",
    "-- RelWithDebInfo/monitor.exe \\\n",
    "-m=\"pedestrian-and-vehicle-detector-adas-0001.bin\" \\\n",
    "-c=\"pedestrian-and-vehicle-detector-adas-0001.xml\" \\\n",
    "-i=\"car-detection.mp4\" \\\n",
    "-cc=0.7 -e=\"b\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Create the Survey analysis report\n",
    "%sx advixe-cl.exe \\\n",
    "--report survey \\\n",
    "--project-dir runs/advise/cpu_run_1 \\\n",
    "--format text \\\n",
    "--report-output=runs/advise/cpu_run_1/cpu_run_1.txt \\\n",
    "-- RelWithDebInfo/monitor.exe \\\n",
    "-m=\"pedestrian-and-vehicle-detector-adas-0001.bin\" \\\n",
    "-c=\"pedestrian-and-vehicle-detector-adas-0001.xml\" \\\n",
    "-i=\"car-detection.mp4\" \\\n",
    "-cc=0.7 -e=\"b\"\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<font color=blue><b>Trip Counts analysis</b></font> - Dynamically identifies the number of times loops and functions are invoked and executed (also called call count/loop count and iteration count respectively). Use Trip Counts data to:\n",
    "* Detect loops with too-small trip counts and trip counts that are not a multiple of vector length.\n",
    "* Analyze parallelism granularity more deeply.\n",
    "\n",
    "<font color=blue><b>FLOP analysis</b></font> - Dynamically measures floating-point and integer operations, and memory traffic. Use the FLOP analysis to generate application memory usage and performance values that help you make better decisions about your vectorization strategy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Trip counts analysis and FLOP analysis\n",
    "!start \"\" cmd /C advixe-cl.exe \\\n",
    "--collect tripcounts -flop -stacks \\\n",
    "--project-dir runs/advise/cpu_run_1 \\\n",
    "-- RelWithDebInfo/monitor.exe \\\n",
    "-m=\"pedestrian-and-vehicle-detector-adas-0001.bin\" \\\n",
    "-c=\"pedestrian-and-vehicle-detector-adas-0001.xml\" \\\n",
    "-i=\"car-detection.mp4\" \\\n",
    "-cc=0.7 -e=\"b\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!start \"\" cmd /C advixe-cl.exe \\\n",
    "--collect map –mark-up-list=1,2,3,4 \\\n",
    "--project-dir runs/advise/cpu_run_1 \\\n",
    "-- RelWithDebInfo/monitor.exe \\\n",
    "-m=\"pedestrian-and-vehicle-detector-adas-0001.bin\" \\\n",
    "-c=\"pedestrian-and-vehicle-detector-adas-0001.xml\" \\\n",
    "-i=\"car-detection.mp4\" \\\n",
    "-cc=0.7 -e=\"b\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!xcopy \"C:\\Program Files (x86)\\IntelSWTools\\Advisor\\pythonapi\\examples\" . /s /e"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Generate a combined report showing all data collected\n",
    "!\"C:\\Program Files (x86)\\IntelSWTools\\Advisor\\advixe-vars.bat\"\n",
    "#!python \"joined.py\" > report.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Generate an html report\n",
    "import subprocess\n",
    "subprocess.call([\"advixe-python.exe\", \"to_html.py runs/advise\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!advixe-python.exe to_html.py runs/advise/cpu_run_1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#You need to do a memory access pattern collection to collect cache statistics\n",
    "!start \"\" cmd /C advixe-cl.exe \\\n",
    "--collect map –mark-up-list=4 \\\n",
    "--project-dir advise1 \\\n",
    "-- RelWithDebInfo/monitor.exe \\\n",
    "-m=\"pedestrian-and-vehicle-detector-adas-0001.bin\" \\\n",
    "-c=\"pedestrian-and-vehicle-detector-adas-0001.xml\" \\\n",
    "-i=\"car-detection.mp4\" \\\n",
    "-cc=0.7 -e=\"b\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#conda install -c blaze sqlite3\n",
    "!advixe-python.exe cache.py runs/advise/cpu_run_1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!advixe-python.exe system_info.py runs/advise/cpu_run_1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%capture\n",
    "#Launch Intel Advisor 2019 GUI\n",
    "!\"C:\\Program Files (x86)\\IntelSWTools\\Advisor 2019\\bin32\\advixe-gui.exe\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Using Intel® Inspector"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Intel® Inspector is a dynamic memory and threading error checking tool for users developing serial and multithreaded applications on Windows* and Linux* operating systems. Follow these steps to get started using the Intel Inspector. <br><br>\n",
    "![alt text](Inspector1.jpg) <br>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!start \"\" cmd /C \"C:\\Program Files (x86)\\IntelSWTools\\Inspector 2019\\inspxe-vars.bat\"\n",
    "#For the command line interface:\n",
    "!start \"\" cmd /C \"C:\\Program Files (x86)\\IntelSWTools\\Inspector 2019\\bin64\\inspxe-cl.exe\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#List all checking modes\n",
    "!inspxe-cl.exe -collect-list"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {},
   "source": [
    "//Build the below programs mm1.cpp and mm2.cpp using VS2017 (E.g., cl /EHsc mm1.cpp)\n",
    "//Incorrect Memory Management\n",
    "//This can occur when we call free() more than once, access memory after freeing it,\n",
    "//or free a memory block that was never allocated as shown in the code below:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```C++\n",
    "//mm1.cpp\n",
    "\n",
    "#include <stdio.h>\n",
    "#include <stdlib.h>\n",
    "#include <string.h>\n",
    "\n",
    "void populate(char **str) {\n",
    "\t// 1. Not OK if later freeing the memory (mm1.cpp)\n",
    "\t*str = \"Memory\";\n",
    "}\n",
    "\n",
    "int main() {\n",
    "\tchar *s;\n",
    "\tpopulate(&s);\n",
    "\tprintf(\"%s\", s);   // should print \"Memory\"\n",
    "\tfree(s);\n",
    "\treturn 0;\n",
    "}\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%sx inspxe-cl -collect mi1 -r runs/inspect/mi1 mm1.exe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%sx inspxe-cl -create-suppression-file \"runs/inspect/mi1/suppressions/mySup\" -r runs/inspect/mi1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```C++\n",
    "//mm2.cpp\n",
    "\n",
    "#include <stdio.h>\n",
    "#include <stdlib.h>\n",
    "#include <string.h>\n",
    "\n",
    "void populate(char **str) {\n",
    "\t// 1. OK (mm1.cpp)\n",
    "\t*str = (char *)malloc(sizeof(char) * 7);\n",
    "\tstrcpy(*str, \"Memory\");\n",
    "}\n",
    "\n",
    "int main() {\n",
    "\tchar *s;\n",
    "\tpopulate(&s);\n",
    "\tprintf(\"%s\", s);   // should print \"Memory\"\n",
    "\tfree(s);\n",
    "\treturn 0;\n",
    "}\n",
    "\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%sx inspxe-cl -collect mi1 -r runs/inspect/mi12 mm2.exe"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```C++\n",
    "// lockGuardDeadlock.cpp\n",
    "\n",
    "#include <iostream>\n",
    "#include <chrono>\n",
    "#include <mutex>\n",
    "#include <thread>\n",
    "\n",
    "\n",
    "int main() {\n",
    "    std::mutex m1;\n",
    "    std::mutex m2;\n",
    "    std::thread t1([&m1, &m2] {\n",
    "        std::cout<<\"\\n1. Acquiring m1.\\n\";\n",
    "        m1.lock();\n",
    "        std::this_thread::sleep_for(std::chrono::milliseconds(10));\n",
    "        std::cout<<\"\\n1. Acquiring m2\\n\";\n",
    "        m2.lock(); \n",
    "    });\n",
    "    std::thread t2([&m1, &m2] {\n",
    "        std::cout<<\"\\n2. Acquiring m2\\n\";\n",
    "        m2.lock();\n",
    "        std::this_thread::sleep_for(std::chrono::milliseconds(10));\n",
    "        std::cout<<\"\\n2. Acquiring m1\\n\";\n",
    "        m1.lock();\n",
    "    });\n",
    "\n",
    "    t1.join();\n",
    "    t2.join();\n",
    "\tstd::cout<<std::endl;\n",
    "}\n",
    "\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!start \"\" cmd /k inspxe-cl -collect ti2 -r runs/inspect/ti2 deadlock.exe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#create supression file\n",
    "%sx inspxe-cl.exe \\\n",
    "-r \"runs/inspect/ti2\" \\\n",
    "-create-suppression-file \"runs\\inspect\\ti2\\suppressions\\mysup\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Reporting: regression status\n",
    "!inspxe-cl -report status –r runs/inspect/ti2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "#Report the observations\n",
    "%sx inspxe-cl -report observations  -r runs/inspect/ti2/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Reporting: problem list\n",
    "!inspxe-cl -report problems –r runs/inspect/ti2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Reporting: extended problem list\n",
    "!inspxe-cl -report problems -report-all –r runs/inspect/ti2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Reporting: other CLI options\n",
    "#XML output\n",
    "!inspxe-cl -report problems –format=xml -r runs/inspect/ti2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#CSV output\n",
    "!inspxe-cl -report problems -format csv -csv-delimiter \" \" -report-output runs/inspect/ti2/observations.csv -r runs/inspect/ti2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import csv\n",
    "import sys\n",
    "with open(\"runs/inspect/ti2/observations.csv\", 'r') as csvfile:\n",
    "\t\treader = csv.reader(csvfile)\n",
    "\t \n",
    "\t\tfor row in reader:\n",
    "\t\t\tprint(row)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Export full result\n",
    "!inspxe-cl -export -archive-name runs/inspect/ti2/ti2.inspxez -include-sources -result-dir runs/inspect/ti2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%capture\n",
    "#Launch Intel Inspector 2019 GUI\n",
    "!\"C:\\Program Files (x86)\\IntelSWTools\\Inspector 2019\\bin64\\inspxe-gui.exe\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#At the end, set the OpenCV tracing flag to zero.\n",
    "%env OPENCV_TRACE=0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}