Optimization of Sums of Bivariate Functions: An Introduction to Relaxation-Based Methods for the Case of Finite Domains

TL;DR

This paper explores the complexity and relaxation-based methods for optimizing sums of bivariate functions over finite domains, introducing tractable formulations and analyzing their limitations through theoretical and experimental insights.

Contribution

It introduces measure-valued relaxations and approximation techniques for sums of bivariates, and investigates their applicability and limitations in optimization.

Findings

NP-equivalence of optimizing sums of bivariates

Tractable formulations via linear programming and closed-form solutions

Experimental validation on various problem classes

Abstract

We study the optimization of functions with $n>2$ arguments that have a representation as a sum of several functions that have only $2$ of the $n$ arguments each, termed sums of bivariates, on finite domains. The complexity of optimizing sums of bivariates is shown to be NP-equivalent and it is shown that there exists free lunch in the optimization of sums of bivariates. Based on measure-valued extensions of the objective function, so-called relaxations, $\ell^2$-approximation, and entropy-regularization, we derive several tractable problem formulations solvable with linear programming, coordinate ascent as well as with closed-form solutions. The limits of applying tractable versions of such relaxations to sums of bivariates are investigated using general results for reconstructing measures from their bivariate marginals. Experiments in which the derived algorithms are applied to random functions, vertex coloring, and signal reconstruction problems provide insights into qualitatively different function classes that can be modeled as sums of bivariates.