Personalization for Web-based Services using Offline Reinforcement Learning

TL;DR

This paper explores offline reinforcement learning to personalize web services, demonstrating significant improvements in user authentication systems by addressing practical challenges and comparing ML techniques.

Contribution

It introduces a practical framework for offline RL in web personalization, including training, evaluation, and deployment insights, with real-world application in a social network.

Findings

Significant improvement in long-term user authentication objectives

Comparison of multiple ML techniques for offline RL

Insights into training and evaluating RL models in production

Abstract

Large-scale Web-based services present opportunities for improving UI policies based on observed user interactions. We address challenges of learning such policies through model-free offline Reinforcement Learning (RL) with off-policy training. Deployed in a production system for user authentication in a major social network, it significantly improves long-term objectives. We articulate practical challenges, compare several ML techniques, provide insights on training and evaluation of RL models, and discuss generalizations.