face-landmark-localization

Overview

This is a Keras implementation of a simple face landmark localization model. It is implemented based on adding the final regression layer to an infamous pretrained networks (provided by Keras).

Dataset

The dataset used here is MUCT face database consisting of 3,755 faces each with 76 hand labeled landmarks. The dataset can be found at: https://github.com/StephenMilborrow/muct. Note that here we resize the original images by a factor of 2 in both width and height (from 640x480 to 320x240) so that my GPU memory is sufficient for the training (1080ti 11GB). The target data (ground truth points) is then also needed to be scaled down by a factor of 2 as well, i.e. (128, 135) to (64, 67.5).

Model

Architecture

The neural network architecture here is ResNet50 with pretrained weights from ImageNet. A global average pooling is added to the net to reduce dimension and the last regression layer with 152 outputs (for each of the 76 landmark points) and with Sigmoid activation function is added.

Loss function

The loss function used in training the mean of Euclidean distance between the ground truth points and the predicted points for each mini-batch. Points which could not be located by human is indicated by coordinate of (0, 0) in the dataset. These points are excluded in the calculation of the loss function.

Training

Dataset is divided into 70% (2628 images) for training and 30% (1127 images) for testing. The ground truth points are first normalized by the width and height of the image before feeding to the network. For the results in the next section, the training parameters are as follows:

• Batch size — 16
• Optimizer — Adam
• Learning rate — 0.001
• number of epochs — 8

To train the model, download the dataset from https://github.com/StephenMilborrow/muct and move all the images from the five folders into the same folder named "images" in the project root. Also, move the "muct76-opencv.csv" file to the project folder as well. The image resizing can be done using resize_images() function in utils.py

PS: The pretrained weights can be found at: https://drive.google.com/file/d/1aEZX1TzmLlsHuN6vLe3neRxlp9_f6486/view?usp=sharing

Result

The results by testing with this dataset could not be shown due to privacy agreement. Results on other dataset will be added later on.