Predicting interviewee attitude and body language from speech descriptors

TL;DR

This study explores how speech acoustic features relate to interviewees' attitudes and body language, finding that acoustic cues can predict perceptions despite topic-induced variations, suggesting multimodal assessment improvements.

Contribution

It demonstrates that acoustic features can predict interviewee perceptions independently of interview topics, highlighting the potential for multimodal analysis in interview assessments.

Findings

Topic influences acoustic parameters significantly.

Acoustic cues can predict perceptions regardless of topic.

Redundancy in acoustic features aids prediction.

Abstract

This present research investigated the relationship between personal impressions and the acoustic nonverbal communication conveyed by employees being interviewed. First, we investigated the extent to which different conversation topics addressed during the interview induced changes in the interviewees' acoustic parameters. Next, we attempted to predict the observed and self-assessed attitudes and body language of the interviewees based on the acoustic data. The results showed that topicality caused significant deviations in the acoustic parameters statistics, but the ability to predict the personal perceptions of the interviewees based on their acoustic non-verbal communication was relatively independent of topicality, due to the natural redundancy inherent in acoustic attributes. Our findings suggest that joint modeling of speech and visual cues may improve the assessment of interviewee profiles.