Skip to content

Gesture Hero is a hand gesture classification system that's build to differentiate between hand gestures representing numbers from 0-5.

License

Notifications You must be signed in to change notification settings

Ahmed-ata112/Gesture_Hero

Repository files navigation

Gesture Hero

1684570250027

Gesture Hero is a hand gesture classification system that's build to differentiate between hand gestures representing numbers from 0-5.

🛠️ Installation Steps:

1. Installing required packages

pip install -r requirements.txt

2. To run for a specific dataset

python ./main.py

💻 Built with

Technologies used in the project:

Jupyter Notebook Python OpenCV Scikit
  • Python
  • scikit-learn
  • OpenCV
  • skimage span

📝 Project description

Gesture Hero is a hand gesture classification system that's build to differentiate between hand gestures representing numbers from 0-5. Gesture Hero is a machine learning based tool, trianed on a dataset with almost 2000 pictures, using an excellent preprocessing, HOG features descriptor and SVM for an accurate answer to your problem.

🧩 Project pipleline

1684571380892

🎞️ Preprocessing

Gesture Hero's strongest point is its preprocessing. By applying classical image processing techniques to preprocess the data for removing shadows, enhancing colors, clipping the area surrounding the hand, rotating so all hands would be pointing in the same direction and resizing for efficiency.

The main preprocessing approach taking is ignoring/ eliminating the channels with misleading information, mainly the ones that represent the illumination.

Segmentation is done using the YCRB channels using a basic thresholding and ignoring the Y(illumination channel), then cropping the photo to only contain the hand by finding the maximum contours surrounding the hand.

To unify the orientations of hands, the preprocessed image is passed to a function to flip it so that the fingers are pointing to the left, based on the histogram of the image, the more dense half represents the palm while the other represents the fingers.

Raw image Preprocessed
1684569675443 1684569885945
1684569753377 1684569906290
1684570066053 1684570109112

🪶 Feature extraction

The main feature descriptor used is the Histogram of Gradients (HOG), since it’s robust to variations in appearance, computationally efficient, its discriminative power as it’s very efficient in capturing the distinguishing features of an object, and lastly its compatibility with machine learning algorithms.

Preprocessed image Visualized HOG
1684570608379 1684570558000
1684570563967 1684570580475

💪🏻 Model training and performance analysis

The chosen model is a support vector machine (SVM) with an RBF kernel, trianed on the HOG extracted features.

Resulting confusion matrix is as follows

1684570926835

Noticing that there's a great confusion between 2,3 and 4 gestures, a 2 layer classification method is used to improve the performance.

The fisrt layer is an SVM model trained on the whole dataset, if the result label is a 2, 3 or 4, the image is then passed to another SVM model that's only trained on the 2,3,4 dataset. This has significantly improved the accuracy of the classificatin.

1684571178913

👥 Collaborators

Sarah Elzayat Ahmed ata abdallah Doaa Magdy Rufaida Kassem

About

Gesture Hero is a hand gesture classification system that's build to differentiate between hand gestures representing numbers from 0-5.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Contributors 4

  •  
  •  
  •  
  •