MTCNN Face Detection and Matching using Facenet Tensorflow

Face Detection and Matching using Facenet Tensorflow

This article is about the comparison of two faces using Facenet python library. Human faces are a unique and beautiful art of nature. It has two eyes with eyebrows, one nose, one mouth and unique structure of face skeleton that affects the structure of cheeks, jaw, and forehead. How do you compare two people by their faces? Actually, we see them and conclude instantly. But how does our brain do it instantly? Actually, our brain is trained naturally in such that it can identify the location of the face and knows what regions of the face are common among all human faces then it only compares those regions outlines in super speed.

A similar phenomenon is done in Facenet python library and we are going to use it to compare two faces. The phenomenon of face matching is split into following steps.

  1. Identify, crop and align face
  2. Generate embeddings
  3. Compare embeddings

Identify, crop and align face

Face detection and alignment are based on the paper “Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Neural Networks” by authors “K. Zhang and Z. Zhang and Z. Li and Y. Qiao”.

The pipeline of the cascaded framework that includes three-stage multi-task deep convolutional networks.

Face Detection facenet pnetFirstly, candidate windows are produced through a fast Proposal Network (P-Net)

 

Face Detection facenet rnet After that, it refines these candidates in the next stage through a Refinement Network (R-Net).

 

Face Detection facenet onet In the third stage, The Output Network (O-Net) produces final bounding box and facial landmarks position.

 

This method of face detection has an advantage on various light condition, face poses variations and visual variations of the face.

Demo Code for face detection using Multi-task Cascaded Convolutional Neural Networks:

Create an empty virtual environment and activate the environment.

git clone https://github.com/arunmandal53/facematch.git
cd facematch
pip install -r requirements.txt

Run the face detection demo:

python face_detect_demo.py --img=images/faces.jpg

You may get like this result.

Face Detection using Facenet in Python - faces

note:- press any key to exit window.

Generate Embeddings

What is embedding?

Face embedding is multidimensional numerical vector representation of a face which represents the unique identity of the face. Facenet used 128 dimensions and created a model that maps any human face in generic. When we provide an input image to the model it gives us 128 bytes of numerical vector data that may be generated by comparison with model mapped generic face representation. These embedding points are easily comparable by measuring Euclidean distance.

Demo Code for generating embedding using pre-trained facenet model

Run the face detection demo:

python face_embeddings_demo.py --img=images/daniel-radcliffe_5.jpg

You will get a result like this:

Face Detection using Facenet in Python - embedding

Compare embeddings

Facenet used methods to directly map facial features into 128 dimensions of numerical data that uniquely define the face and it can be compared with other faces by using Euclidean distance with the following formula.

Face Detection using Facenet in Python - euclidean

You can see in the code how numpy is used to calculate euclidean distance.

Demo Code for comparison of two faces by calculating Euclidean distance between their embeddings 

Note:- Threshold is currently used by 1.10 you can change according to your need.

Run the face detection demo:

python face_match_demo.py --img1=images/daniel-radcliffe_2.jpg --img2=images/daniel-radcliffe_4.jpg

You may a result like:

Face Detection using Facenet in Python - daniel-radcliffe_4Face Detection using Facenet in Python - daniel-radcliffe_2

Daniel Radcliffe

 (Age 28) – (Age 11)

distance = 0.94406
Result = same person

Some other examples:

Face Detection using Facenet in Python - amitabh_young

Face Detection using Facenet in Python - amitabh_old

Amitabh Bachchan
 (Age 48) –  (Age 70)
distance = 0.93345463
Result = same person

 

Face Detection using Facenet in Python - Barack_Obama

Face Detection using Facenet in Python - IlhamAnas

Barack Obama – Ilham Anas
distance = 1.2404629
Result = not the same person

 

Face Detection using Facenet in Python - dicaprio-the-titanic

Face Detection using Facenet in Python - dicaprio-the-revenant

Leonardo Dicaprio
 (clean face) – (not clean face)
distance = 0.98249304
Result = same person

 

Face Detection using Facenet in Python - angelina-jolie-young

Face Detection using Facenet in Python - Esha-gupta

Angelina Jolie – Esha Gupta
distance = 1.1594397
Result = not the same person

Summary

We can see that Facenet is doing good to correctly generate embedding of a face in such a way that by calculating Euclidean distance between embedding of two faces gives the correct result. We need to correctly set up the threshold according to our need. Higher the threshold more risk of getting the false result. In above demo the distance threshold 1.10 to make a decision for face matching.

You might also want to check out another article that we have, How to make your own face recognition app using OpenCV library, here.

References

Facenet Research Paper: https://arxiv.org/abs/1503.03832
MTCNN Research Paper: https://arxiv.org/abs/1604.02878

6 Comments on MTCNN Face Detection and Matching using Facenet Tensorflow

  1. Hey this does work. But for around 25% of the inputs I try with I get a distance of -1. Are there any specifications on the type of images I use?

    • You will get -1 if one of image for comparison does not contain any face. Increase factor to 0.809 or tweak other parameters.

1 Trackbacks & Pingbacks

  1. How to use MTCNN face detection to monitor your background | Python 3.6

Leave a Reply

Your email address will not be published.




This site uses Akismet to reduce spam. Learn how your comment data is processed.