Towards Social & Engaging Peer Learning: Predicting Backchanneling and Disengagement in Children

TL;DR

This paper develops models to predict children's engagement and backchanneling behavior in social robot interactions, using multimodal features and time series analysis to enhance naturalness and trust in peer learning scenarios.

Contribution

It introduces novel multimodal features and time series modeling approaches for predicting engagement and backchanneling in children interacting with robots.

Findings

Multimodal features like pupil dilation and blink rate improve predictions.

Time series dynamics are key predictors of engagement.

Socio-demographic factors influence backchanneling behavior.

Abstract

Social robots and interactive computer applications have the potential to foster early language development in young children by acting as peer learning companions. However, studies have found that children only trust robots which behave in a natural and interpersonal manner. To help robots come across as engaging and attentive peer learning companions, we develop models to predict whether the listener will lose attention (Listener Disengagement Prediction, LDP) and the extent to which a robot should generate backchanneling responses (Backchanneling Extent Prediction, BEP) in the next few seconds. We pose LDP and BEP as time series classification problems and conduct several experiments to assess the impact of different time series characteristics and feature sets on the predictive performance of our model. Using statistics & machine learning, we also examine which socio-demographic factors influence the amount of time children spend backchanneling and listening to their peers. To lend interpretability to our models, we also analyzed critical features responsible for their predictive performance. Our experiments revealed the utility of multimodal features such as pupil dilation, blink rate, head movements, facial action units which have never been used before. We also found that the dynamics of time series features are rich predictors of listener disengagement and backchanneling.