Approximation of optimization problems with constraints through kernel Sum-Of-Squares

TL;DR

This paper introduces a unified framework using kernel Sum-Of-Squares to approximate constrained optimization problems in infinite-dimensional spaces, providing convergence guarantees and addressing high-dimensional sampling challenges.

Contribution

It presents a general theorem for convergence of kernel Sum-Of-Squares approximations and demonstrates how to handle constraints efficiently in high dimensions.

Findings

Proves convergence guarantees for kSoS approximations.

Shows mitigation of the curse of dimensionality in sampling.

Illustrates application in learning invariant vector fields.

Abstract

Handling an infinite number of inequality constraints in infinite-dimensional spaces occurs in many fields, from global optimization to optimal transport. These problems have been tackled individually in several previous articles through kernel Sum-Of-Squares (kSoS) approximations. We propose here a unified theorem to prove convergence guarantees for these schemes. Pointwise inequalities are turned into equalities within a class of nonnegative kSoS functions. Assuming further that the functions appearing in the problem are smooth, focusing on pointwise equality constraints enables the use of scattering inequalities to mitigate the curse of dimensionality in sampling the constraints. Our approach is illustrated in learning vector fields with side information, here the invariance of a set.