Combining Offline Causal Inference and Online Bandit Learning for Data Driven Decision

TL;DR

This paper introduces a unified framework combining offline causal inference with online bandit learning to improve decision-making using both logged and streaming data, with theoretical guarantees and empirical validation.

Contribution

It presents novel algorithms that integrate offline and online methods, along with the first regret bounds for forest-based bandit algorithms, enhancing decision accuracy.

Findings

Algorithms outperform methods using only logged or online data.

First upper regret bound established for forest-based bandit algorithms.

Empirical results on real datasets demonstrate superior performance.

Abstract

A fundamental question for companies with large amount of logged data is: How to use such logged data together with incoming streaming data to make good decisions? Many companies currently make decisions via online A/B tests, but wrong decisions during testing hurt users' experiences and cause irreversible damage. A typical alternative is offline causal inference, which analyzes logged data alone to make decisions. However, these decisions are not adaptive to the new incoming data, and so a wrong decision will continuously hurt users' experiences. To overcome the aforementioned limitations, we propose a framework to unify offline causal inference algorithms (e.g., weighting, matching) and online learning algorithms (e.g., UCB, LinUCB). We propose novel algorithms and derive bounds on the decision accuracy via the notion of "regret". We derive the first upper regret bound for forest-based online bandit algorithms. Experiments on two real datasets show that our algorithms outperform other algorithms that use only logged data or online feedbacks, or algorithms that do not use the data properly.