Minimization Problems Based on Relative $\alpha$-Entropy II: Reverse Projection

TL;DR

This paper explores reverse minimization problems based on relative $ ext{I}_ ext{alpha}$-entropy, establishing orthogonality properties and transforming complex problems into simpler quasiconvex minimizations for robust estimation and compression.

Contribution

It introduces the concept of reverse $ ext{I}_ ext{alpha}$-projections, proves their orthogonality with linear families, and transforms them into forward projections for easier solutions.

Findings

Orthogonality of power-law and linear families established.

Reverse projections can be converted into forward projections.

Simplifies complex minimization problems to quasiconvex optimization.

Abstract

In part I of this two-part work, certain minimization problems based on a parametric family of relative entropies (denoted $\mathscr{I}_{\alpha}$) were studied. Such minimizers were called forward $\mathscr{I}_{\alpha}$-projections. Here, a complementary class of minimization problems leading to the so-called reverse $\mathscr{I}_{\alpha}$-projections are studied. Reverse $\mathscr{I}_{\alpha}$-projections, particularly on log-convex or power-law families, are of interest in robust estimation problems ($\alpha >1$) and in constrained compression settings ($\alpha <1$). Orthogonality of the power-law family with an associated linear family is first established and is then exploited to turn a reverse $\mathscr{I}_{\alpha}$-projection into a forward $\mathscr{I}_{\alpha}$-projection. The transformed problem is a simpler quasiconvex minimization subject to linear constraints.