Online Orthogonal Matching Pursuit
El Mehdi Saad, Gilles Blanchard, Sylvain Arlot

TL;DR
This paper introduces Online Orthogonal Matching Pursuit, an online algorithm for support recovery in sparse linear regression that balances sample efficiency and computational complexity with proven guarantees.
Contribution
The paper proposes a novel online greedy algorithm for feature selection in sparse linear models, with theoretical analysis of its guarantees and computational complexity.
Findings
Algorithm achieves support recovery with theoretical guarantees.
Computational complexity is analyzed and shown to be efficient.
Supports online feature selection in high-dimensional settings.
Abstract
Greedy algorithms for feature selection are widely used for recovering sparse high-dimensional vectors in linear models. In classical procedures, the main emphasis was put on the sample complexity, with little or no consideration of the computation resources required. We present a novel online algorithm: Online Orthogonal Matching Pursuit (OOMP) for online support recovery in the random design setting of sparse linear regression. Our procedure selects features sequentially, alternating between allocation of samples only as needed to candidate features, and optimization over the selected set of variables to estimate the regression coefficients. Theoretical guarantees about the output of this algorithm are proven and its computational complexity is analysed.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsSparse and Compressive Sensing Techniques · Machine Learning and ELM · Face and Expression Recognition
MethodsFeature Selection
