Exploring Semi-Supervised Learning for Predicting Listener Backchannels

TL;DR

This paper introduces a semi-supervised method for predicting listener backchannels in conversational agents, reducing manual annotation needs while maintaining high accuracy and naturalness in responses.

Contribution

The study presents a semi-supervised approach for backchannel prediction that achieves comparable performance to manual annotations, enhancing scalability in conversational agent development.

Findings

Semi-supervised labels achieve 95% of manual annotation performance.

60% of users found semi-supervised predicted responses more natural.

Personality influences backchannel signal types.

Abstract

Developing human-like conversational agents is a prime area in HCI research and subsumes many tasks. Predicting listener backchannels is one such actively-researched task. While many studies have used different approaches for backchannel prediction, they all have depended on manual annotations for a large dataset. This is a bottleneck impacting the scalability of development. To this end, we propose using semi-supervised techniques to automate the process of identifying backchannels, thereby easing the annotation process. To analyze our identification module's feasibility, we compared the backchannel prediction models trained on (a) manually-annotated and (b) semi-supervised labels. Quantitative analysis revealed that the proposed semi-supervised approach could attain 95% of the former's performance. Our user-study findings revealed that almost 60% of the participants found the backchannel responses predicted by the proposed model more natural. Finally, we also analyzed the impact of personality on the type of backchannel signals and validated our findings in the user-study.