Difference of Submodular Minimization via DC Programming

TL;DR

This paper explores the minimization of the difference of two submodular functions by leveraging the connection to difference of convex functions, introducing variants of the DC algorithm with improved convergence and stronger guarantees, and demonstrating superior performance in applications.

Contribution

The paper introduces variants of the DC algorithm and its complete form for DS minimization, extending convergence analysis and providing stronger local minimality guarantees.

Findings

Proposed algorithms outperform existing baselines in speech corpus and feature selection.

Extended convergence properties of DCA are established and connected to DS problem guarantees.

Stronger local minimality guarantees are achieved with CDCA.

Abstract

Minimizing the difference of two submodular (DS) functions is a problem that naturally occurs in various machine learning problems. Although it is well known that a DS problem can be equivalently formulated as the minimization of the difference of two convex (DC) functions, existing algorithms do not fully exploit this connection. A classical algorithm for DC problems is called the DC algorithm (DCA). We introduce variants of DCA and its complete form (CDCA) that we apply to the DC program corresponding to DS minimization. We extend existing convergence properties of DCA, and connect them to convergence properties on the DS problem. Our results on DCA match the theoretical guarantees satisfied by existing DS algorithms, while providing a more complete characterization of convergence properties. In the case of CDCA, we obtain a stronger local minimality guarantee. Our numerical results show that our proposed algorithms outperform existing baselines on two applications: speech corpus selection and feature selection.