Multi-objective Optimization of Notifications Using Offline Reinforcement Learning

TL;DR

This paper presents an offline reinforcement learning framework for optimizing multi-objective notification decisions in mobile systems, addressing challenges like distributional shift and demonstrating improved performance through experiments.

Contribution

It introduces a novel offline RL approach using Conservative Q-learning for multi-objective notification optimization in mobile systems.

Findings

Improved notification decision quality in offline and online tests

Effective mitigation of distributional shift in offline RL

Enhanced user engagement through optimized notifications

Abstract

Mobile notification systems play a major role in a variety of applications to communicate, send alerts and reminders to the users to inform them about news, events or messages. In this paper, we formulate the near-real-time notification decision problem as a Markov Decision Process where we optimize for multiple objectives in the rewards. We propose an end-to-end offline reinforcement learning framework to optimize sequential notification decisions. We address the challenge of offline learning using a Double Deep Q-network method based on Conservative Q-learning that mitigates the distributional shift problem and Q-value overestimation. We illustrate our fully-deployed system and demonstrate the performance and benefits of the proposed approach through both offline and online experiments.