Streaming Methods for Restricted Strongly Convex Functions with Applications to Prototype Selection

TL;DR

This paper introduces streaming algorithms for restricted-strongly-convex and weakly submodular functions, enabling efficient data summarization with theoretical guarantees and practical effectiveness on real datasets.

Contribution

It establishes constant factor approximation guarantees for streaming algorithms on RSC and RSM functions, a subclass of weakly submodular functions, and demonstrates their efficiency in data summarization.

Findings

Algorithms are significantly faster than existing methods.

Achieve comparable solution quality to state-of-the-art algorithms.

Effective on large datasets like MNIST and UCI alphabet dataset.

Abstract

In this paper, we show that if the optimization function is restricted-strongly-convex (RSC) and restricted-smooth (RSM) -- a rich subclass of weakly submodular functions -- then a streaming algorithm with constant factor approximation guarantee is possible. More generally, our results are applicable to any monotone weakly submodular function with submodularity ratio bounded from above. This (positive) result which provides a sufficient condition for having a constant factor streaming guarantee for weakly submodular functions may be of special interest given the recent negative result (Elenberg et al., 2017) for the general class of weakly submodular functions. We apply our streaming algorithms for creating compact synopsis of large complex datasets, by selecting $m$ representative elements, by optimizing a suitable RSC and RSM objective function. Above results hold even with additional constraints such as learning non-negative weights, for interpretability, for each selected element indicative of its importance. We empirically evaluate our algorithms on two real datasets: MNIST- a handwritten digits dataset and Letters- a UCI dataset containing the alphabet written in different fonts and styles. We observe that our algorithms are orders of magnitude faster than the state-of-the-art streaming algorithm for weakly submodular functions and with our main algorithm still providing equally good solutions in practice.