Subspace Selection via DR-Submodular Maximization on Lattices

TL;DR

This paper explores subspace selection problems in machine learning, characterizing when greedy algorithms are effective by introducing directional DR-submodular functions and demonstrating their applicability to PCA and dictionary selection.

Contribution

The paper introduces directional DR-submodular functions on lattices, providing a theoretical framework for understanding greedy algorithm effectiveness in subspace selection problems.

Findings

Directional DR-submodular functions characterize problem approximability.

Greedy algorithms achieve provable approximation guarantees for these functions.

Applications include PCA and sparse dictionary selection.

Abstract

The subspace selection problem seeks a subspace that maximizes an objective function under some constraint. This problem includes several important machine learning problems such as the principal component analysis and sparse dictionary selection problem. Often, these problems can be solved by greedy algorithms. Here, we are interested in why these problems can be solved by greedy algorithms, and what classes of objective functions and constraints admit this property. To answer this question, we formulate the problems as optimization problems on lattices. Then, we introduce a new class of functions, directional DR-submodular functions, to characterize the approximability of problems. We see that the principal component analysis, sparse dictionary selection problem, and these generalizations have directional DR-submodularities. We show that, under several constraints, the directional DR-submodular function maximization problem can be solved efficiently with provable approximation factors.