Greedy Minimization of Weakly Supermodular Set Functions

TL;DR

This paper introduces weak-$eta$-supermodularity for set functions and demonstrates that greedy algorithms with an extension phase can effectively approximate solutions for various machine learning optimization problems under constraints.

Contribution

It defines weak-$eta$-supermodularity and proves that greedy extension strategies yield near-optimal solutions for constrained minimization problems.

Findings

Greedy extension improves approximation guarantees for weakly supermodular functions.

New bicriteria results for k-means, sparse regression, and column subset selection.

Framework applies to multiple machine learning optimization objectives.

Abstract

This paper defines weak-$\alpha$-supermodularity for set functions. Many optimization objectives in machine learning and data mining seek to minimize such functions under cardinality constrains. We prove that such problems benefit from a greedy extension phase. Explicitly, let $S^*$ be the optimal set of cardinality $k$ that minimizes $f$ and let $S_0$ be an initial solution such that $f(S_0)/f(S^*) \le \rho$. Then, a greedy extension $S \supset S_0$ of size $|S| \le |S_0| + \lceil \alpha k \ln(\rho/\varepsilon) \rceil$ yields $f(S)/f(S^*) \le 1+\varepsilon$. As example usages of this framework we give new bicriteria results for $k$-means, sparse regression, and columns subset selection.