Exploring Local Norms in Exp-concave Statistical Learning

TL;DR

This paper establishes a new excess risk bound for stochastic convex optimization with exp-concave losses, leveraging local norms and geometric assumptions, advancing theoretical understanding in statistical learning.

Contribution

It provides a novel $O( d / n + rac{ ext{log}(1/ ext{delta})}{n} )$ excess risk bound for exp-concave losses, answering a key open question.

Findings

Derived a bound valid for a wide class of bounded exp-concave losses.

Introduced a unified geometric assumption on gradients and local norms.

Enhanced theoretical guarantees in stochastic convex optimization.

Abstract

We consider the problem of stochastic convex optimization with exp-concave losses using Empirical Risk Minimization in a convex class. Answering a question raised in several prior works, we provide a $O( d / n + \log( 1 / \delta) / n )$ excess risk bound valid for a wide class of bounded exp-concave losses, where $d$ is the dimension of the convex reference set, $n$ is the sample size, and $\delta$ is the confidence level. Our result is based on a unified geometric assumption on the gradient of losses and the notion of local norms.