Provable Non-Convex Optimization and Algorithm Validation via Submodularity

TL;DR

This paper explores continuous submodularity, extending submodular properties to non-convex functions, and develops algorithms with strong guarantees for optimization and validation in various applications like revenue maximization and probabilistic models.

Contribution

It introduces the concept of continuous submodularity, characterizes a broad class of non-convex functions, and proposes algorithms with approximation guarantees for their maximization.

Findings

Continuous submodular functions encompass many practical applications.

Algorithms for maximizing continuous submodular functions achieve strong approximation guarantees.

Information-theoretic analysis reveals robustness of MaxCut algorithms.

Abstract

Submodularity is one of the most well-studied properties of problem classes in combinatorial optimization and many applications of machine learning and data mining, with strong implications for guaranteed optimization. In this thesis, we investigate the role of submodularity in provable non-convex optimization and validation of algorithms. A profound understanding which classes of functions can be tractably optimized remains a central challenge for non-convex optimization. By advancing the notion of submodularity to continuous domains (termed "continuous submodularity"), we characterize a class of generally non-convex and non-concave functions -- continuous submodular functions, and derive algorithms for approximately maximizing them with strong approximation guarantees. Meanwhile, continuous submodularity captures a wide spectrum of applications, ranging from revenue maximization with general marketing strategies, MAP inference for DPPs to mean field inference for probabilistic log-submodular models, which renders it as a valuable domain knowledge in optimizing this class of objectives. Validation of algorithms is an information-theoretic framework to investigate the robustness of algorithms to fluctuations in the input/observations and their generalization ability. We investigate various algorithms for one of the paradigmatic unconstrained submodular maximization problem: MaxCut. Due to submodularity of the MaxCut objective, we are able to present efficient approaches to calculate the algorithmic information content of MaxCut algorithms. The results provide insights into the robustness of different algorithmic techniques for MaxCut.