Thompson Sampling for Repeated Newsvendor

TL;DR

This paper analyzes the effectiveness of Thompson Sampling in the repeated newsvendor problem, providing regret bounds, insights into exploration-exploitation, and demonstrating superior performance over existing methods through simulations.

Contribution

It offers the first regret analysis of Thompson Sampling for the repeated newsvendor model, extending to general parametric distributions and providing practical insights.

Findings

Thompson Sampling achieves near-optimal regret bounds.

TS automatically balances exploration and exploitation based on order size.

Numerical results show TS outperforms existing approaches.

Abstract

In this paper, we investigate the performance of Thompson Sampling (TS) for online learning with censored feedback, focusing primarily on the classic repeated newsvendor model--a foundational framework in inventory management--and demonstrating how our techniques can be naturally extended to a broader class of problems. We first model demand using a Weibull distribution and initialize TS with a Gamma prior to dynamically adjust order quantities. Our analysis establishes optimal (up to logarithmic factors) frequentist regret bounds for TS without imposing restrictive prior assumptions. More importantly, it yields novel and highly interpretable insights on how TS addresses the exploration-exploitation trade-off in the repeated newsvendor setting. Specifically, our results show that when past order quantities are sufficiently large to overcome censoring, TS accurately estimates the unknown demand parameters, leading to near-optimal ordering decisions. Conversely, when past orders are relatively small, TS automatically increases future order quantities to gather additional demand information. Then, we extend our analysis to general parametric distribution family and provide proof for Bayesian regret. Extensive numerical simulations further demonstrate that TS outperforms more conservative and widely-used approaches such as online convex optimization, upper confidence bounds, and myopic Bayesian dynamic programming.