Feature selection in weakly coherent matrices

TL;DR

This paper introduces a perturbation bound for the smallest singular value of matrices with low coherence, enabling a fast feature extraction algorithm by selecting submatrices with desirable spectral properties.

Contribution

It presents a novel perturbation bound for the smallest singular value in weakly coherent matrices and develops a corresponding efficient feature extraction algorithm.

Findings

The perturbation bound effectively predicts singular value changes after column addition.

The proposed algorithm efficiently extracts features with controlled spectral properties.

Results demonstrate improved speed and accuracy in feature selection tasks.

Abstract

A problem of paramount importance in both pure (Restricted Invertibility problem) and applied mathematics (Feature extraction) is the one of selecting a submatrix of a given matrix, such that this submatrix has its smallest singular value above a specified level. Such problems can be addressed using perturbation analysis. In this paper, we propose a perturbation bound for the smallest singular value of a given matrix after appending a column, under the assumption that its initial coherence is not large, and we use this bound to derive a fast algorithm for feature extraction.