Lesson 13 : Face Detection

Face Detection

In the past few years, face recognition has become one of the most promising applications of computer vision. Face detection can be considered to be a substantial part of face recognition operations.

The method of face detection in pictures is complicated because, well, human faces are widely different from one another! They can have different poses, expressions, position, and orientation, skin color, have glasses or facial hair or not, etc. Then, there are also differences in camera gain, lighting conditions, and image resolution.

Face detection is the action of locating human faces in an image and optionally returning different kinds of face-related data.

How Do We Detect Faces?

There are a few methods that you can use to detect face such as:

1. Knowledge-Based
2. Feature-Based

Knowledge-Based Detection

The knowledge-based method depends on a set of rules and is based on human knowledge to detect the faces.

E.g. A face often appears with two eyes that are symmetric to each other, a nose and a mouth.

Pros

1. Easy to come up with simple rules.
2. Based on the coded rules, facial features in an input image are extracted first, and face candidates are identified.

Cons

1. Difficult to translate human knowledge into rules precisely.
   1. Detailed rules fail to detect faces, like the rule that, ‘A person must have two eyes’: According to this rule, no face will be identified in this image.
   2. General rules may find many false positives: Like there is an eye in a face. According to this, the face will be identified in this tattoo of a similar-looking eye.
2. This approach alone is insufficient and unable to find many faces in multiple images.

Feature-Based Detection

Feature-based detection uses what are known as face landmarks to detect faces.

Face landmarks are a set of easy-to-find points on a face, such as the pupils or the tip of the nose.

By default, there are 68 predefined landmark points. The following image shows all landmark points:

The feature-based method detects faces by extracting the face landmarks of the face.

Pros

1. Features are invariant to pose and orientation change.
2. This method has a higher chance of face detection.

Cons

1. Difficult to locate facial features due to noise in the images.
2. Difficult to detect features in complex backgrounds.

Face Detection Blocks in PictoBlox

The Face Detection extension in PictoBlox has blocks dedicated to face detection.

Let’s first add the extension to our project:

Alert: Make sure you have an active internet connection to load the model.

1. Create a new project in PictoBlox.
2. Next, click on the Add Extension button and add the Face Detection extension.
   
3. The face detection models will be downloaded which may take few seconds to load.
   
4. Once downloaded you will get a message saying “Model loaded successfully”.

Facial Detection Block

To execute face detection, we will use analyse image from () block.

You can input the image in the following ways:

1. Camera feed
2. Stage

This block. analyses the image and saves the face information locally, which can be accessed using other blocks similar to computer vision.

You have to run this block every time you want to analyze a new image from the camera or stage.

Controlling camera feed

You can control the camera feed on the stage using turn () video on stage with () % transparency block.

You can change the following parameters of the block:

1. Camera state:
   1. OFF – The video feed on the stage will stop.
   2. ON – The video feed on the stage will start.
   3. ON flipped – The video feed on the stage will start with the video mirrored. So, your right hand will be shown as left hand and vice versa.
2. Transparency: This parameter makes the video translucent. O is defined as the camera feed will be shown on the stage and 100 is the video will fade out completely from the stage.

Edit the parameter and try out this block yourself.

Example – Analyzing Image on Stage

In this topic, we will analyze the following image:

1. Download the image from here: https://ai.thestempedia.com/wp-content/uploads/2022/02/Boy-and-Girl.jpg
2. In PictoBlox, turn off the camera using turn () video on stage with () % transparency block, and choosing the options as shown. Click on the block to execute it.
   
3. Next upload it as backdrop.
   
4. Select Tobi sprite and hide it.
   
5. You will find the stage like this:
   
6. Drag analyse image from () block, change the feed as stage and click on the block to analyze the image.
   
7. Now we will see what things we get when we analyze images in face detection.

Get # faces

Once you have analyzed the images, you can use the get # faces block to get the number of faces recognized in the image.

For the image being analyzed, you will get 2.

Get expression of face ()

The get expression of face () block reports the recognized expression of the selected face. If the face selected is not detected then it reports NULL.

Example: For the first face in the image being analyzed, the expression is Happy.

Is expression of face () ()

The is expression of face () () block checks whether the selected face has a particular emotion or not. If the emotion matches, then the block returns true, else false.

E.g., for face 1 in the image, the emotion is Happy.

Get () of face ()

The get () of face () block reports the following attributes of the image:

1. X position: X coordinate of the center of the image on the stage
2. Y position: Y coordinate of the center of the image on the stage
3. Width: Width of the face
4. Height: Height of the face

Note: This block reports an accurate position when the image is analyzed from stage or camera feed.

Using this block you can make the bounding box for the face like this:

We will learn how to do the same in the next topic by making a face detector.

Get () position of landmark () of face ()

The get () position of landmark () of face () block reports the x and y position of the selected landmark (number from 1 to 68) of the selected face.

Using this block you can draw the landmark points on the stage like this:

Conclusion

We have looked at the various blocks to analyze images of faces and get key information out of them. In the next lesson, will see how to draw the bounding box on the detected face and report their expressions.