Which Channel to Ask My Question? Personalized Customer Service Request Stream Routing using Deep Reinforcement Learning

TL;DR

This paper introduces a deep reinforcement learning framework for personalized customer service routing, optimizing resource use and customer satisfaction across multiple communication channels.

Contribution

It presents a novel deep reinforcement learning approach, specifically the PER-DoDDQN method, for dynamic customer service request routing, outperforming existing systems.

Findings

The proposed framework outperforms current production systems.

PER-DoDDQN surpasses other deep Q-learning variants.

The method achieves an optimal balance between resources and customer satisfaction.

Abstract

Customer services are critical to all companies, as they may directly connect to the brand reputation. Due to a great number of customers, e-commerce companies often employ multiple communication channels to answer customers' questions, for example, chatbot and hotline. On one hand, each channel has limited capacity to respond to customers' requests, on the other hand, customers have different preferences over these channels. The current production systems are mainly built based on business rules, which merely considers tradeoffs between resources and customers' satisfaction. To achieve the optimal tradeoff between resources and customers' satisfaction, we propose a new framework based on deep reinforcement learning, which directly takes both resources and user model into account. In addition to the framework, we also propose a new deep-reinforcement-learning based routing method-double dueling deep Q-learning with prioritized experience replay (PER-DoDDQN). We evaluate our proposed framework and method using both synthetic and a real customer service log data from a large financial technology company. We show that our proposed deep-reinforcement-learning based framework is superior to the existing production system. Moreover, we also show our proposed PER-DoDDQN is better than all other deep Q-learning variants in practice, which provides a more optimal routing plan. These observations suggest that our proposed method can seek the trade-off where both channel resources and customers' satisfaction are optimal.