Face Recognition Using MTCNN, VGG-Face2 and Pinecone DB

An implementation of siamease neural networks on one shot learning tasks for face recognition tasks utilising MTCNN, FaceNet and Pinecone DB for building an interactive and easy to use application to store and detect faces from images as well as camera inputs accuratly.

Deployment

To use this project application run

  https://face-recognition-tf-1.onrender.com

Note : Loading may take some time due to limited server processing capability.

Outputs

Key Features

1) Face detection using MTCNN :
Used MTCNN for accurate face detection on images.

2) Embedding extraction using FacNet :
created embeddings vectors of shape (512,1) using a Tensorflow pretrained model on face datasets.

3) Pinecone Database for efficent storage and retreival:
Created a pinecone DB index for storing embdeeings by creating relevent metadata and upserting it to the index with cosine similarity as serach parameter.

4) Siamease Network Architehture:
Implemented a Siamease Network like architehture to acheive one shot learning for face recognition using a combination of face detection and face recognition.

5) Streamlit application:
Created and deployed an interactive streamlit application to interact with the project. Deployment was done on render.

6) Multiface Detection capabilities:
The model is able to detect and recognise multiple faces in an image however for creating a new entry in the database for a person, an induidual image is required to ensure integrity of the data in the database.

7) Modularity of code:
The project is cretated so that the induvidual blocks can be changed to suit the detection needs, eg. MTCNN can be replace with YOLO detection for faster results for applications such as ANPR after replacing FaceNet with a suitably trained model to generate embeddings.

Pinecone API Reference

Refer to Pinecone Documentation at : pc docs

Pinecone env variables used :

Parameter	Type	Description
Pinecone key	string
Pinecone index	string	Name of Index

Installing pinecone client

pip install pinecone --upgrade pinecone-client

Connecting to index and upserting

from pinecone import Pinecone
pc = Pinecone(os.environ["PINECONE_API_KEY"])
index = pc.Index(os.environ["PINECONE_IDX"])

vectors = 'Create Vectors'
index.upsert(vectors)

Query top k vectors

out = index.query(
      vector = vectors.tolist(),
      top_k = k,
      include_metadata = True
  )

Dependencies

• OpenCV
• Tensorflow
• MTCNN
• FaceNet
• Pinecone
• streamlit
• os
• dotenv
• numpy

Installation :

After pulling this repo, run:

pip install requirements

env file setup :

setup an env file to store api keys and index information as the template :

pinecone_key = "your_api_key"
pinecone_index = "your_index_name"