Optimal Online Generalized Linear Regression with Stochastic Noise and Its Application to Heteroscedastic Bandits

TL;DR

This paper analyzes online generalized linear regression with stochastic noise, providing near-optimal regret bounds and applying the results to heteroscedastic bandits with variance-aware regret guarantees.

Contribution

It offers a sharp regret analysis for FTRL in noisy generalized linear models and extends to heteroscedastic bandits with variance-aware bounds.

Findings

Regret upper bound of O(σ^2 d log T) + o(log T) for stochastic noise.

Lower bound of Ω(σ^2 d log(T/d)) showing near-optimality.

First variance-aware regret bound for heteroscedastic generalized linear bandits.

Abstract

We study the problem of online generalized linear regression in the stochastic setting, where the label is generated from a generalized linear model with possibly unbounded additive noise. We provide a sharp analysis of the classical follow-the-regularized-leader (FTRL) algorithm to cope with the label noise. More specifically, for $\sigma$-sub-Gaussian label noise, our analysis provides a regret upper bound of $O(\sigma^2 d \log T) + o(\log T)$, where $d$ is the dimension of the input vector, $T$ is the total number of rounds. We also prove a $\Omega(\sigma^2d\log(T/d))$ lower bound for stochastic online linear regression, which indicates that our upper bound is nearly optimal. In addition, we extend our analysis to a more refined Bernstein noise condition. As an application, we study generalized linear bandits with heteroscedastic noise and propose an algorithm based on FTRL to achieve the first variance-aware regret bound.