Enhancing Public Speaking Skills in Engineering Students Through AI

TL;DR

This paper presents an AI-driven multi-modal assessment system that provides personalized feedback on public speaking skills in engineering students by analyzing speech, facial expressions, gestures, and coherence, aiming to improve communication training.

Contribution

The study develops a novel multi-modal AI model that fuses verbal and non-verbal cues to deliver scalable, personalized feedback for public speaking training, surpassing previous separate assessment systems.

Findings

AI feedback moderately aligns with expert evaluations

Gemini Pro outperforms other LLMs in assessment accuracy

The system enables repeated, scalable practice for students

Abstract

This research-to-practice full paper was inspired by the persistent challenge in effective communication among engineering students. Public speaking is a necessary skill for future engineers as they have to communicate technical knowledge with diverse stakeholders. While universities offer courses or workshops, they are unable to offer sustained and personalized training to students. Providing comprehensive feedback on both verbal and non-verbal aspects of public speaking is time-intensive, making consistent and individualized assessment impractical. This study integrates research on verbal and non-verbal cues in public speaking to develop an AI-driven assessment model for engineering students. Our approach combines speech analysis, computer vision, and sentiment detection into a multi-modal AI system that provides assessment and feedback. The model evaluates (1) verbal communication (pitch, loudness, pacing, intonation), (2) non-verbal communication (facial expressions, gestures, posture), and (3) expressive coherence, a novel integration ensuring alignment between speech and body language. Unlike previous systems that assess these aspects separately, our model fuses multiple modalities to deliver personalized, scalable feedback. Preliminary testing demonstrated that our AI-generated feedback was moderately aligned with expert evaluations. Among the state-of-the-art AI models evaluated, all of which were Large Language Models (LLMs), including Gemini and OpenAI models, Gemini Pro emerged as the best-performing, showing the strongest agreement with human annotators. By eliminating reliance on human evaluators, this AI-driven public speaking trainer enables repeated practice, helping students naturally align their speech with body language and emotion, crucial for impactful and professional communication.