Adaptive Minimax Regret against Smooth Logarithmic Losses over High-Dimensional $\ell_1$-Balls via Envelope Complexity

TL;DR

This paper introduces the envelope complexity framework to analyze minimax regret for logarithmic loss over high-dimensional $ ext{l}_1$-balls, deriving a Bayesian predictor with a novel spike-and-tails prior that adaptively achieves near-optimal regret.

Contribution

It proposes the envelope complexity framework and a new spike-and-tails prior to adaptively attain minimax regret in high-dimensional logarithmic loss scenarios.

Findings

The regret bound depends mainly on smoothness and radius, with logarithmic factors.

The spike-and-tails prior outperforms traditional minimax priors in preliminary tests.

The framework generalizes existing regret bounds to high-dimensional settings.

Abstract

We develop a new theoretical framework, the \emph{envelope complexity}, to analyze the minimax regret with logarithmic loss functions and derive a Bayesian predictor that adaptively achieves the minimax regret over high-dimensional $\ell_1$-balls within a factor of two. The prior is newly derived for achieving the minimax regret and called the \emph{spike-and-tails~(ST) prior} as it looks like. The resulting regret bound is so simple that it is completely determined with the smoothness of the loss function and the radius of the balls except with logarithmic factors, and it has a generalized form of existing regret/risk bounds. In the preliminary experiment, we confirm that the ST prior outperforms the conventional minimax-regret prior under non-high-dimensional asymptotics.