Theory and applications of the Sum-Of-Squares technique

TL;DR

This paper reviews the Sum-of-Squares (SOS) technique, a method transforming complex optimization problems into semidefinite programs, with extensions to infinite-dimensional spaces and applications in information theory.

Contribution

It provides an overview of SOS, including its extension to kernel methods and its use in estimating information-theoretic quantities.

Findings

SOS transforms non-convex problems into semidefinite programs

Extension of SOS to infinite-dimensional feature spaces using kernels

Application of SOS in estimating log-partition functions

Abstract

The Sum-of-Squares (SOS) approximation method is a technique used in optimization problems to derive lower bounds on the optimal value of an objective function. By representing the objective function as a sum of squares in a feature space, the SOS method transforms non-convex global optimization problems into solvable semidefinite programs. This note presents an overview of the SOS method. We start with its application in finite-dimensional feature spaces and, subsequently, we extend it to infinite-dimensional feature spaces using reproducing kernels (k-SOS). Additionally, we highlight the utilization of SOS for estimating some relevant quantities in information theory, including the log-partition function.