Network Flow Algorithms for Structured Sparsity

TL;DR

This paper introduces a novel network flow-based optimization method for structured sparsity-inducing norms with overlapping groups, enabling scalable solutions for large-scale learning problems.

Contribution

It establishes a connection between the structured sparsity norm and network flow optimization, providing an efficient polynomial-time algorithm for overlapping groups.

Findings

Algorithm scales to millions of variables

Effective for image and video data applications

Solves the proximal problem exactly in polynomial time

Abstract

We consider a class of learning problems that involve a structured sparsity-inducing norm defined as the sum of $\ell_\infty$-norms over groups of variables. Whereas a lot of effort has been put in developing fast optimization methods when the groups are disjoint or embedded in a specific hierarchical structure, we address here the case of general overlapping groups. To this end, we show that the corresponding optimization problem is related to network flow optimization. More precisely, the proximal problem associated with the norm we consider is dual to a quadratic min-cost flow problem. We propose an efficient procedure which computes its solution exactly in polynomial time. Our algorithm scales up to millions of variables, and opens up a whole new range of applications for structured sparse models. We present several experiments on image and video data, demonstrating the applicability and scalability of our approach for various problems.