The core work of a face recognition algorithm is to identify the location of a face from a picture。The algorithm of recognition can be said to be various, and I will introduce them to you one by one.

 

1.HoG Face Detection

The algorithm uses a traditional machine learning algorithm to recognize faces. The traditional machine learning algorithm is characterized by manually constructing features and then feeding the constructed features into the model for training.

The algorithm uses HoG to extract face features from images and SVM algorithm to classify them.

HoG (Histogram of Oriented Gradient) feature is a feature descriptor used in computer vision and image processing for object detection, which constitutes features by computing and counting the gradient direction histogram of local regions of an image.

The implementation of the algorithm is available in the dlib library, and we take a look at the core code below

import dlib
# Load the pre-trained HoG face detector
hog_face_detector = dlib.get_frontal_face_detector()

# Perform face detection on images
results = hog_face_detector(imgRGB, 0)

for bbox in results:
  x1 = bbox.left()  # x coordinate of the upper left corner of the face
  y1 = bbox.top()  # y coordinate of the upper left corner of the face
  x2 = bbox.right()  # x coordinate of the lower right corner of the face
  y2 = bbox.bottom()  # y coordinate of the lower right corner of the face

results stores multiple faces detected in a single image, and iterates through results to get a rectangular box for each face.

The test example is as follows：

The green boxed out is the face detected by the algorithm.

HoG face detection is relatively high performance because it uses traditional machine learning algorithm, and it can run relatively fast on CPU. However, it cannot detect faces smaller than 80*80, and the recognition effect is not good for rotated faces and non-frontal faces.

2.Deep learning face detection

While traditional machine learning algorithms detect faster, the accuracy leaves much to be desired. Deep learning based face detection algorithms tend to be more accurate.

Presented here is the use of the residual network ResNet-10 to detect multiple faces in a single channel (Single Shot Detector, SSD) of an image through the network (model). It is called SSD algorithm for short.

First, the original image needs to be blob preprocessed and then fed directly to the model for detection

The cv2 library provides the implementation of this algorithm with the following core code：

import cv2

# Load the pre-trained SSD model
opencv_dnn_model = cv2.dnn.readNetFromCaffe(
      prototxt="models/deploy.prototxt"
      , caffeModel="models/res10_300x300_ssd_iter_140000_fp16.caffemodel")

# Raw image blob processing
preprocessed_image = cv2.dnn.blobFromImage(image, scalefactor=1.0, size=(300, 300), mean=(104.0, 117.0, 123.0), swapRB=False, crop=False)
 
# blob image feeding into the model
opencv_dnn_model.setInput(preprocessed_image)

# Model inference for face detection
results = opencv_dnn_model.forward()  

# Traversing the face
for face in results[0][0]:
  # Confidence
  face_confidence = face[2]
  
  # Coordinate points of the upper left and lower right corners of the face border
  x1 = int(bbox[0] * image_width)
  y1 = int(bbox[1] * image_height)
  x2 = int(bbox[2] * image_width)
  y2 = int(bbox[3] * image_height)

The results[0][0] store the detected multiple faces, each face is expressed in an array, and the 3rd bit of the array stores the confidence level, which can be used to filter the unconfident faces by a threshold value. Bits 4 to 7 of the array store the coordinates of the top left and bottom right corners of the detected face rectangle.

Compared with HoG face detection, the SSD algorithm can also detect obscured, non-frontal faces.

 

3.Convolutional neural network face detection

Convolution is not much to say, those who understand computer vision know.

The dlib library provides an implementation of the convolutional neural network face detection algorithm, similar in usage to the previous one

import dlib

# Documenting pre-trained models
cnn_face_detector = dlib.cnn_face_detection_model_v1("models/mmod_human_face_detector.dat")

# Face Detection
results = cnn_face_detector(imgRGB, 0)

# Iterate through each face
for face in results:
  # Face border      
  bbox = face.rect
  
  # Coordinate points of the upper left and lower right corners of the face border
  x1 = int(bbox.left() * (width/new_width))
  y1 = int(bbox.top() * (height/new_height))
  x2 = int(bbox.right() * (width/new_width))
  y2 = int(bbox.bottom() * (height/new_height))

The resolution of the results is similar to the above and will not be repeated here.

The face detection algorithm using convolutional neural network has obvious advantages, it is more accurate and robust than the first two, and it can also detect faces under occlusion.

Even non-frontal and poorly lit images can be detected very well

However, the corresponding drawback of this algorithm is obvious: the detection process takes a very long time and cannot run in real time on the CPU.

4.BlazeFace

The above algorithms either have high accuracy and slow speed, or high speed and low accuracy. So is there a detection algorithm that has both high accuracy and high performance?

The answer is yes, BlazeFace is a very lightweight and highly accurate face detector, claimed to be a sub-millisecond face detector. It is inspired by the Single Shot MultiBox Detector (SSD) and MobileNetv2.

The Mediapipe library provides an implementation of this algorithm with the following core code：

import mediapipe as mp

# Drawing Tools
mp_drawing = mp.solutions.drawing_utils

# Initialize face detection model
mp_face_detection = mp.solutions.face_detection
mp_face_detector = mp_face_detection.FaceDetection(min_detection_confidence=0.4)

results = mp_face_detector.process(imgRGB)

if results.detections:

  # Face change detection of the face
  for face_no, face in enumerate(results.detections):

      # Key points for painting a human face
      mp_drawing.draw_detection(image=output_image, detection=face, keypoint_drawing_spec=mp_drawing.DrawingSpec(color=(0,255,0),thickness=-1, circle_radius=image_width//115), bbox_drawing_spec=mp_drawing.DrawingSpec(color=(0,255,0),thickness=image_width//180))

      # Face painting frames
      face_bbox = face.location_data.relative_bounding_box
      x1 = int(face_bbox.xmin*image_width)
      y1 = int(face_bbox.ymin*image_height)

      cv2.rectangle(output_image, pt1=(x1, y1-image_width//20), pt2=(x1+image_width//16, y1), color=(0, 255, 0), thickness=-1)

The effect is as follows：

As you can see, BlazeFace algorithm not only detects faces, but also recognizes 6 key points of faces (eyes, nose, ears, mouth).

Above are the 4 algorithms of face recognition shared today.

After recognizing the face, it is very simple to do face attendance, we can Embedding the face into a vector and calculate the distance between the vectors.