Optimization and Sampling Under Continuous Symmetry: Examples and Lie Theory

TL;DR

This paper explores how continuous symmetries, described by Lie theory, can be leveraged to develop efficient algorithms for nonconvex optimization and sampling problems, with broad applications in science and machine learning.

Contribution

It introduces the use of Lie theory to connect nonconvex symmetric manifolds with convex polytopes, enabling new optimization and sampling techniques.

Findings

Reduction of linear optimization over Lie group orbits to polytopes

Application of Kostant's convexity theorem for optimization

Use of HCIZ formulas for efficient sampling over Lie groups

Abstract

In the last few years, the notion of symmetry has provided a powerful and essential lens to view several optimization or sampling problems that arise in areas such as theoretical computer science, statistics, machine learning, quantum inference, and privacy. Here, we present two examples of nonconvex problems in optimization and sampling where continuous symmetries play -- implicitly or explicitly -- a key role in the development of efficient algorithms. These examples rely on deep and hidden connections between nonconvex symmetric manifolds and convex polytopes, and are heavily generalizable. To formulate and understand these generalizations, we then present an introduction to Lie theory -- an indispensable mathematical toolkit for capturing and working with continuous symmetries. We first present the basics of Lie groups, Lie algebras, and the adjoint actions associated with them, and we also mention the classification theorem for Lie algebras. Subsequently, we present Kostant's convexity theorem and show how it allows us to reduce linear optimization problems over orbits of Lie groups to linear optimization problems over polytopes. Finally, we present the Harish-Chandra and the Harish-Chandra--Itzykson--Zuber (HCIZ) formulas, which convert partition functions (integrals) over Lie groups into sums over the corresponding (discrete) Weyl groups, enabling efficient sampling algorithms.