# Sequential Counterfactual Risk Minimization

## TL;DR

This paper extends Counterfactual Risk Minimization to a sequential setting where policies can be deployed multiple times, introducing a new estimator and demonstrating improved theoretical and empirical performance.

## Contribution

It proposes Sequential Counterfactual Risk Minimization (SCRM), extending CRM theory to multiple deployments and introducing a novel estimator for better performance.

## Key findings

- Improved excess risk and regret rates with multiple deployments
- Empirical validation in discrete and continuous action spaces
- Demonstrated benefits of multiple policy deployments

## Abstract

Counterfactual Risk Minimization (CRM) is a framework for dealing with the logged bandit feedback problem, where the goal is to improve a logging policy using offline data. In this paper, we explore the case where it is possible to deploy learned policies multiple times and acquire new data. We extend the CRM principle and its theory to this scenario, which we call "Sequential Counterfactual Risk Minimization (SCRM)." We introduce a novel counterfactual estimator and identify conditions that can improve the performance of CRM in terms of excess risk and regret rates, by using an analysis similar to restart strategies in accelerated optimization methods. We also provide an empirical evaluation of our method in both discrete and continuous action settings, and demonstrate the benefits of multiple deployments of CRM.