Contextual Bandit Applications in Customer Support Bot

TL;DR

This paper explores the deployment of contextual bandit algorithms in Microsoft's customer support virtual agent, demonstrating significant improvements in resolution rates and reduction in escalations through real-world implementations.

Contribution

It presents practical implementations of neural-linear and multi-armed bandit algorithms for intent disambiguation and recommendations in a production support bot, validated by A/B testing.

Findings

Over 12% increase in problem resolution rate

Over 4% decrease in escalations to human operators

Successful deployment of bandit algorithms in real-world customer support

Abstract

Virtual support agents have grown in popularity as a way for businesses to provide better and more accessible customer service. Some challenges in this domain include ambiguous user queries as well as changing support topics and user behavior (non-stationarity). We do, however, have access to partial feedback provided by the user (clicks, surveys, and other events) which can be leveraged to improve the user experience. Adaptable learning techniques, like contextual bandits, are a natural fit for this problem setting. In this paper, we discuss real-world implementations of contextual bandits (CB) for the Microsoft virtual agent. It includes intent disambiguation based on neural-linear bandits (NLB) and contextual recommendations based on a collection of multi-armed bandits (MAB). Our solutions have been deployed to production and have improved key business metrics of the Microsoft virtual agent, as confirmed by A/B experiments. Results include a relative increase of over 12% in problem resolution rate and relative decrease of over 4% in escalations to a human operator. While our current use cases focus on intent disambiguation and contextual recommendation for support bots, we believe our methods can be extended to other domains.