Challenges Faced by Facial Recognition System

Date: August 18, 2020

Author: Kajal Mishra

Face Recognition has always been one of the most fascinating and intriguing technologies as it deals with human faces. Covid-19 outbreak has propelled the world to move towards touchless facial recognition technology. It is gaining huge traction worldwide owing to its contactless biometric features. Companies are getting rid of traditional fingerprinting scanners and creating massive business opportunities by adopting AI-based facial recognition technology. Some of the applications where its usage has become crucial are security & surveillance, authentication/access control systems, digital healthcare, photo retrieval, etc.

As said, opportunities and challenges go hand in hand. Growing commercial interest for face recognition is encouraging, but it also turns out to be a challenging endeavour when it comes to problems associated which have played continuously with its quality of delivery. These challenges arise when the situations are non-cooperative and causes the varied facial appearance/expressions.

Listed below are the challenges which limit the potential of a Facial RecognitionSystem to go that extra mile.


Illumination stands for light variations.  The slight change in lighting conditions cause a significant challenge for automated face recognition and can have a significant impact on its results. If the illumination tends to vary, the same individual gets captured with the same sensor and with an almost identical facial expression and pose, the results that emerge may appear quite different.

Illumination changes the face appearance drastically. It has been found that the difference between two same faces with different illuminations is higher than two different faces taken under same illumination.


Facial Recognition Systems are highly sensitive to pose variations. The pose of a face varies when the head movement and viewing angle of the person changes. The movements of head or differing POV of a camera can invariably cause changes in face appearance and generate intra‐class variations making automated face recognition rates drop drastically. It becomes a challenge to identify the real face when the rotation angle goes higher. It may result in faulty recognition or no recognition if the database only has the frontal view of the face.


Occlusion means blockage, and it occurs when one or other parts of the face are blocked and whole face is not available as an input image. Occlusion is considered one of the most critical challenges in face recognition system.

It occurs due to beard, moustache, accessories (goggle, cap, mask, etc.), and it is prevalent in real-world scenario. The presence of such components makes the subject diverse and hence making automated face recognition process a tough nut to crack.


Face is one of the most crucial biometrics as its unique features play a crucial role in providing human identity and emotions. Varying situations cause different moods which result in showing various emotions and eventually change in facial expressions.

Different expressions of the same individual are another significant factor that needs to be taken into account. Human expressions are particularly macro-expressions which are happiness, sadness, anger, disgust, fear, surprise. Micro-expressions are the one which shows the rapid facial patterns and happen involuntarily.

Macro and micro expressions find their place on someone's face due to changes in one's emotional state and in the wake of such emotions- which are many- the efficient recognition becomes difficult.

Low Resolution

The minimum resolution for any standard image should be 16*16. The picture with the resolution less than 16*16 is called the low resolution image. These low resolution images can be found through small scale standalone cameras like CCTV cameras in streets, ATM cameras, supermarket security cameras. These cameras can capture a small part of the human face area and as the camera is not very close to face, they can only capture the face region of less than 16*16. Such a low resolution image doesn’t provide much information as most of them are lost. It can be a big challenge in the process of recognizing the faces.


Face appearance/texture changes over a period of time and reflect as ageing, which is yet another challenge in facial recognition system. With the increasing age, the human face features, shapes/lines, and other aspects also change. It is done for visual observation and image retrieval after a long period.

For accuracy checking, the dataset for a different age group of people over a period of time is calculated. Here, the recognition process depends on feature extraction, basic features like wrinkles, marks, eyebrows, hairstyles, etc.

Model Complexity

Existing state-of-the-art facial recognition methods rely on ‘too-deep’ Convolutional Neural Network (CNN) architecture which is very complex and unsuitable for real-time performance on embedded devices.

An ideal face recognition system should be tolerant of variations in illumination, expression, pose, and occlusion. It should be scalable to a large number of users with a need for capturing minimal images during registration while doing away with complex architecture at the same time.


Face is the most essential part of the human body, and its unique features make it even more crucial in identifying humans. Various algorithms and technologies are used worldwide to make the face recognition process more accurate and reliable. The applications of this ever-growing technology are also expanding in healthcare, security, defence, forensic, and transportation, requiring more accuracy. However, some challenges are ubiquitous while developing face recognition technology such as pose, occlusion, expressions, ageing, etc, which have been discussed above in the article.

If you are interested in discovering how a low-complex, embedded friendly CNN architecture can be used for face recognition, an FR model that is independent of image variations and real-time on embedded devices, please write a mail to us at

By submitting this form, you authorize PathPartner to contact you with further information about our relevant content, products and services. You may unsubscribe any time. We are committed to your privacy. For more details, refer our Privacy Policy

Camera & IoT

By submitting this form, you authorize PathPartner to contact you with further information about our relevant content, products and services. You may unsubscribe any time. We are committed to your privacy. For more details, refer our Privacy Policy

Notify of
Inline Feedbacks
View all comments
Back to Top