Autonomous CRM Control via CLV Approximation with Deep Reinforcement Learning in Discrete and Continuous Action Space

TL;DR

This paper presents a deep reinforcement learning framework for autonomous CRM control that estimates customer lifetime value and optimizes marketing actions in both discrete and continuous spaces.

Contribution

It introduces a novel approach combining CLV approximation with deep Q-learning for personalized marketing decision-making.

Findings

Effective CLV estimation from client state space.

Optimal marketing actions derived for individual clients.

Validated on KDD Cup 1998 dataset with promising results.

Abstract

The paper outlines a framework for autonomous control of a CRM (customer relationship management) system. First, it explores how a modified version of the widely accepted Recency-Frequency-Monetary Value system of metrics can be used to define the state space of clients or donors. Second, it describes a procedure to determine the optimal direct marketing action in discrete and continuous action space for the given individual, based on his position in the state space. The procedure involves the use of model-free Q-learning to train a deep neural network that relates a client's position in the state space to rewards associated with possible marketing actions. The estimated value function over the client state space can be interpreted as customer lifetime value, and thus allows for a quick plug-in estimation of CLV for a given client. Experimental results are presented, based on KDD Cup 1998 mailing dataset of donation solicitations.