Estimating Presentation Competence using Multimodal Nonverbal Behavioral Cues

TL;DR

This study explores how multimodal nonverbal cues like facial expressions, body pose, and audio features can be used to automatically assess presentation competence, aiding skill development.

Contribution

It introduces a multimodal approach combining visual and audio cues for automated presentation competence estimation using machine learning.

Findings

Achieved 71.25% accuracy with early fusion on the same dataset.

Reaching 78.11% accuracy with late fusion across datasets.

Fusion strategies improved the performance of competence estimation.

Abstract

Public speaking and presentation competence plays an essential role in many areas of social interaction in our educational, professional, and everyday life. Since our intention during a speech can differ from what is actually understood by the audience, the ability to appropriately convey our message requires a complex set of skills. Presentation competence is cultivated in the early school years and continuously developed over time. One approach that can promote efficient development of presentation competence is the automated analysis of human behavior during a speech based on visual and audio features and machine learning. Furthermore, this analysis can be used to suggest improvements and the development of skills related to presentation competence. In this work, we investigate the contribution of different nonverbal behavioral cues, namely, facial, body pose-based, and audio-related features, to estimate presentation competence. The analyses were performed on videos of 251 students while the automated assessment is based on manual ratings according to the T\"ubingen Instrument for Presentation Competence (TIP). Our classification results reached the best performance with early fusion in the same dataset evaluation (accuracy of 71.25%) and late fusion of speech, face, and body pose features in the cross dataset evaluation (accuracy of 78.11%). Similarly, regression results performed the best with fusion strategies.