Submodularity in Action: From Machine Learning to Signal Processing Applications

TL;DR

This paper explores the role of submodularity in signal processing and machine learning, demonstrating how its properties enable scalable, efficient optimization with provable guarantees in various real-world applications.

Contribution

It introduces submodular-friendly applications, clarifies the link between submodularity and convexity/concavity, and provides practical algorithms with theoretical performance bounds.

Findings

Submodularity enables scalable optimization in large-scale problems.

The paper presents real-world case studies demonstrating practical benefits.

Algorithms with provable worst-case guarantees are developed for various applications.

Abstract

Submodularity is a discrete domain functional property that can be interpreted as mimicking the role of the well-known convexity/concavity properties in the continuous domain. Submodular functions exhibit strong structure that lead to efficient optimization algorithms with provable near-optimality guarantees. These characteristics, namely, efficiency and provable performance bounds, are of particular interest for signal processing (SP) and machine learning (ML) practitioners as a variety of discrete optimization problems are encountered in a wide range of applications. Conventionally, two general approaches exist to solve discrete problems: $(i)$ relaxation into the continuous domain to obtain an approximate solution, or $(ii)$ development of a tailored algorithm that applies directly in the discrete domain. In both approaches, worst-case performance guarantees are often hard to establish. Furthermore, they are often complex, thus not practical for large-scale problems. In this paper, we show how certain scenarios lend themselves to exploiting submodularity so as to construct scalable solutions with provable worst-case performance guarantees. We introduce a variety of submodular-friendly applications, and elucidate the relation of submodularity to convexity and concavity which enables efficient optimization. With a mixture of theory and practice, we present different flavors of submodularity accompanying illustrative real-world case studies from modern SP and ML. In all cases, optimization algorithms are presented, along with hints on how optimality guarantees can be established.