Rewarding Chatbots for Real-World Engagement with Millions of Users
Robert Irvine, Douglas Boubert, Vyas Raina, Adian Liusie, Ziyi Zhu,, Vineet Mudupalli, Aliaksei Korshuk, Zongyi Liu, Fritz Cremer, Valentin, Assassi, Christie-Carol Beauchamp, Xiaoding Lu, Thomas Rialan, William, Beauchamp

TL;DR
This paper presents a reward-based training method for social chatbots that significantly improves user engagement and retention by using human feedback and automatic pseudo-labels, validated through large-scale A/B testing.
Contribution
It introduces a novel reward modeling approach using pseudo-labels from user interactions to enhance chatbot engagement and retention.
Findings
Mean conversation length increased by up to 70%.
User retention improved by over 30%.
Effective use of human feedback for training chatbots.
Abstract
The emergence of pretrained large language models has led to the deployment of a range of social chatbots for chitchat. Although these chatbots demonstrate language ability and fluency, they are not guaranteed to be engaging and can struggle to retain users. This work investigates the development of social chatbots that prioritize user engagement to enhance retention, specifically examining the use of human feedback to efficiently develop highly engaging chatbots. The proposed approach uses automatic pseudo-labels collected from user interactions to train a reward model that can be used to reject low-scoring sample responses generated by the chatbot model at inference time. Intuitive evaluation metrics, such as mean conversation length (MCL), are introduced as proxies to measure the level of engagement of deployed chatbots. A/B testing on groups of 10,000 new daily chatbot users on the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAI in Service Interactions · Digital Mental Health Interventions · Recommender Systems and Techniques
