The Effect of Multiple Replies for Natural Language Generation Chatbots

TL;DR

This study investigates how providing multiple replies in NLG chatbots enhances user experience by creating a group chat atmosphere, showing that multiple replies improve engagement and satisfaction over single replies.

Contribution

The paper introduces a novel approach of using multiple replies in NLG chatbots to improve user experience, supported by experimental evidence comparing different reply and avatar conditions.

Findings

Multiple replies lead to better user engagement.

Users prefer receiving five replies over a single reply.

Avatar type influences user perception, with multiple replies compensating for anonymous avatars.

Abstract

In this research, by responding to users' utterances with multiple replies to create a group chat atmosphere, we alleviate the problem that Natural Language Generation chatbots might reply with inappropriate content, thus causing a bad user experience. Because according to our findings, users tend to pay attention to appropriate replies and ignore inappropriate replies. We conducted a 2 (single reply vs. five replies) x 2 (anonymous avatar vs. anime avatar) repeated measures experiment to compare the chatting experience in different conditions. The result shows that users will have a better chatting experience when receiving multiple replies at once from the NLG model compared to the single reply. Furthermore, according to the effect size of our result, to improve the chatting experience for NLG chatbots which is single reply and anonymous avatar, providing five replies will have more benefits than setting an anime avatar.