Learning Exponential Families in High-Dimensions: Strong Convexity and Sparsity

TL;DR

This paper investigates the strong convexity properties of exponential families in high-dimensional settings, providing theoretical insights into their generalization capabilities and analysis under L1 regularization.

Contribution

It characterizes a strong convexity property of exponential families, enabling analysis of high-dimensional learning with sparsity and L1 regularization.

Findings

Established a strong convexity property for exponential families.

Quantified generalization ability in high-dimensional models.

Analyzed L1 regularization effects on exponential family learning.

Abstract

The versatility of exponential families, along with their attendant convexity properties, make them a popular and effective statistical model. A central issue is learning these models in high-dimensions, such as when there is some sparsity pattern of the optimal parameter. This work characterizes a certain strong convexity property of general exponential families, which allow their generalization ability to be quantified. In particular, we show how this property can be used to analyze generic exponential families under L_1 regularization.