Occlusion Aware Student Emotion Recognition based on Facial Action Unit Detection

TL;DR

This paper introduces an occlusion-aware facial action unit extraction model using attention mechanisms to improve emotion recognition accuracy in classroom settings, especially under partial occlusion conditions.

Contribution

It proposes a novel occlusion-aware architecture with attention and adaptive learning for facial action unit detection, enhancing emotion recognition robustness.

Findings

The model effectively handles partial occlusion in facial images.

Attention mechanisms improve AU detection accuracy.

Enhanced reliability of emotion analysis in classroom environments.

Abstract

Given that approximately half of science, technology, engineering, and mathematics (STEM) undergraduate students in U.S. colleges and universities leave by the end of the first year [15], it is crucial to improve the quality of classroom environments. This study focuses on monitoring students' emotions in the classroom as an indicator of their engagement and proposes an approach to address this issue. The impact of different facial parts on the performance of an emotional recognition model is evaluated through experimentation. To test the proposed model under partial occlusion, an artificially occluded dataset is introduced. The novelty of this work lies in the proposal of an occlusion-aware architecture for facial action units (AUs) extraction, which employs attention mechanism and adaptive feature learning. The AUs can be used later to classify facial expressions in classroom settings. This research paper's findings provide valuable insights into handling occlusion in analyzing facial images for emotional engagement analysis. The proposed experiments demonstrate the significance of considering occlusion and enhancing the reliability of facial analysis models in classroom environments. These findings can also be extended to other settings where occlusions are prevalent.