Facial Emotion Recognition Through Detection of Facial Action Units and Their Intensity

Rohan Appasaheb Borgalli, Sunil Surve
2022 Scientific Visualization  
Facial expression recognition (FER) is a vital process in many applications in computer vision, medical, human-computer interface, video games, AI, E-education, security, distance psychotherapy, and Counselling. In the past, only basic emotions are the focus of research but now, with advancement in technology and need along with basic emotion, reorganization of compound emotions is also getting important. But, recognizing facial expressions with accuracy is challenging. To deal with that,
more » ... action units for detecting appropriate basic and compound facial emotion are useful. Among state-of-the-art methods for FER systems, detection of facial action units (AUs) showed good results. Combining these AUs to detect particular basic and compound facial expressions is key and improves the accuracy of basic and compound facial expressions. We have used standard CNN architecture VGGNet, XceptionNet, and ResNet to conduct experiments in the proposed method to explore CNN architecture that best performs for action unit recognition. We modified them slightly by changing a few final fully connected layers to detect facial action unit intensity. Using this modified architecture for CK+, MMI, DISFA, and DISFA+ database that detects action units with good accuracy, it is intern mapped to find basic and compound facial emotions. Our Proposed method achieves an overall accuracy for Action Unit detection using Xceptionnet network for MMI & DISFA are, giving promising results average F1-score is 72% and 74%, respectively. In contrast, a network for CK+ and DISFA+ has an overall F1-score is 62% for both.
doi:10.26583/sv.14.1.06 fatcat:qttguvfh65htnkj7cuwb4cpei4