Facial Expression Recognition on Video and Image 📷

INTRODUCTION

An emotion classification project which takes image files captured from human face and predicts the emotion conveyed from the facial expression. Please visit the application here to explore the project and its capabilities including real-time video emotion classification and real-time graphs.

DATASETS USED

1. FER2013
   

   The data consists of 48x48 pixel grayscale images of faces. The faces have been automatically registered so that the face is more or less centered and occupies about the same amount of space in each image. The seven categories in the dataset are: (0=Angry, 1=Disgust, 2=Fear, 3=Happy, 4=Sad, 5=Surprise, 6=Neutral). The data set has three columns: "emotion", "pixels", and "Usage". The "emotion" column contains a numeric code ranging from 0 to 6, inclusive, for the emotion that is present in the image. The "pixels" column contains a string surrounded in quotes for each image. The contents of this string a space-separated pixel values in row major order. The "Usage" column indicates which images are for Training(80%), Public Test(10%), and Private Test(10%). This dataset ontains 35,887 images. This dataset was prepared by Pierre-Luc Carrier and Aaron Courville, as part of an ongoing research project. They have graciously provided the workshop organizers with a preliminary version of their dataset to use for this contest.

2. The Extended Cohn-Kanade Dataset (CK+)
   

   This dataset contains 920 images from 920 original CK+ dataset. Data is already reshaped to 48x48 pixels, in grayscale format and facecropped using haarcascade_frontalface_default. Noisy (based on room light/hair format/skin colour) images were adapted to be clearly identified using Haar classifier. Columns from file are defined as emotion/pixels/Usage. Emotions are defined as:

   0 : Anger (45 samples)

   1 : Disgust (59 samples)

   2 : Fear (25 samples)

   3 : Happiness (69 samples)

   4 : Sadness (28 samples)

   5 : Surprise (83 samples)

   6 : Neutral (593 samples)

   7 : Contempt (18 samples)

   Pixels contains 2304 pixels (48x48) each row.

   Usage is determined as Training(80%) / PublicTest(10%) / PrivateTest(10%).

   This dataset was first developed to make a comparison to [ROHIT VERMA - FER2013].

TECHNOLOGIES

1. Tensorflow
2. Keras
3. Scikit-Learn
4. Numpy
5. Pandas
6. Imbalanced-Learn
7. Plotly
8. Matplotlib
9. Seaborn
10. Jupyter Notebook
11. OpenCV
12. Python
13. Flask
14. HTML
15. CSS
16. JavaScript
17. Heroku
18. Kaggle

UI

NOTEBOOK VIEWER LINK

You can check the Jupyter Notebook for all the workings of the model here.

Report

You can find the report of the project here.

SETUP INSTRUCTIONS

You can view all the setup instructions depening on your operating system in the setup folder here.

CONTRIBUTION AND FEEDBACK

If you would like to contribute or have any feedback for this project please feel free to contact me on LinkedIn.

CODE LICENSE

MIT License

Copyright (c) 2023 Georgios Ioannou

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.