Learning to Optimize via Wasserstein Deep Inverse Optimal Control

TL;DR

This paper introduces a novel Wasserstein-based inverse optimal control framework that models user behavior as reinforcement learning, leading to improved performance in social science applications like recommender systems.

Contribution

It presents a unified KL framework and a two-step Wasserstein inverse optimal control method, combining mass transport and GANs for better learning of user cost functions.

Findings

Significant performance improvements over existing methods.

Effective modeling of user behavior in social systems.

Successful application to recommender systems and social networks.

Abstract

We study the inverse optimal control problem in social sciences: we aim at learning a user's true cost function from the observed temporal behavior. In contrast to traditional phenomenological works that aim to learn a generative model to fit the behavioral data, we propose a novel variational principle and treat user as a reinforcement learning algorithm, which acts by optimizing his cost function. We first propose a unified KL framework that generalizes existing maximum entropy inverse optimal control methods. We further propose a two-step Wasserstein inverse optimal control framework. In the first step, we compute the optimal measure with a novel mass transport equation. In the second step, we formulate the learning problem as a generative adversarial network. In two real world experiments - recommender systems and social networks, we show that our framework obtains significant performance gains over both existing inverse optimal control methods and point process based generative models.