Real-Time Confidence Detection through Facial Expressions and Hand Gestures

TL;DR

This paper introduces a real-time facial orientation detection system using Media Pipe Face Mesh, achieving 90% accuracy in head pose estimation to enhance virtual interaction and engagement monitoring.

Contribution

It presents a novel real-time head orientation detection method leveraging facial landmarks and Euler angles, improving accuracy and responsiveness in virtual environments.

Findings

Achieves 90% accuracy in head orientation detection.

Effective in monitoring engagement during online assessments.

Reliable for real-time virtual interaction analysis.

Abstract

Real-time face orientation recognition is a cutting-edge technology meant to track and analyze facial movements in virtual environments such as online interviews, remote meetings, and virtual classrooms. As the demand for virtual interactions grows, it becomes increasingly important to measure participant engagement, attention, and overall interaction. This research presents a novel solution that leverages the Media Pipe Face Mesh framework to identify facial landmarks and extract geometric data for calculating Euler angles, which determine head orientation in real time. The system tracks 3D facial landmarks and uses this data to compute head movements with a focus on accuracy and responsiveness. By studying Euler angles, the system can identify a user's head orientation with an accuracy of 90\%, even at a distance of up to four feet. This capability offers significant enhancements for monitoring user interaction, allowing for more immersive and interactive virtual ex-periences. The proposed method shows its reliability in evaluating participant attentiveness during online assessments and meetings. Its application goes beyond engagement analysis, potentially providing a means for improving the quality of virtual communication, fostering better understanding between participants, and ensuring a higher level of interaction in digital spaces. This study offers a basis for future developments in enhancing virtual user experiences by integrating real-time facial tracking technologies, paving the way for more adaptive and interactive web-based platform.