Reweighted information inequalities

TL;DR

This paper develops a new framework for understanding inequalities related to mixture distributions, providing insights into Fisher information bounds and their implications for Langevin Monte Carlo in multimodal settings.

Contribution

It introduces a variant of log-Sobolev and transport-information inequalities tailored for mixture distributions, linking Fisher information proximity to entropy and transport distances.

Findings

Establishes new inequalities for mixture distributions.

Provides a reinterpretation of Fisher information bounds.

Explains observed phenomena in Langevin Monte Carlo analysis.

Abstract

We establish a variant of the log-Sobolev and transport-information inequalities for mixture distributions. If a probability measure $\pi$ can be decomposed into components that individually satisfy such inequalities, then any measure $\mu$ close to $\pi$ in relative Fisher information is close in relative entropy or transport distance to a reweighted version of $\pi$ with the same mixture components but possibly different weights. This provides a user-friendly interpretation of Fisher information bounds for non-log-concave measures and explains phenomena observed in the analysis of Langevin Monte Carlo for multimodal distributions.