Lessons from Contextual Bandit Learning in a Customer Support Bot

TL;DR

This paper shares practical insights from applying contextual bandits to enhance a customer support chatbot, highlighting challenges faced and solutions developed in real-world natural language processing tasks.

Contribution

It provides empirical lessons and practical guidance on implementing contextual bandits in customer support applications, with insights applicable to similar reinforcement learning scenarios.

Findings

Improved key business metrics for Microsoft Virtual Agent

Identified common challenges in applying CBs to NLP tasks

Proposed practical solutions for RL application issues

Abstract

In this work, we describe practical lessons we have learned from successfully using contextual bandits (CBs) to improve key business metrics of the Microsoft Virtual Agent for customer support. While our current use cases focus on single step einforcement learning (RL) and mostly in the domain of natural language processing and information retrieval we believe many of our findings are generally applicable. Through this article, we highlight certain issues that RL practitioners may encounter in similar types of applications as well as offer practical solutions to these challenges.