Sparse Representation-Based Classification: Orthogonal Least Squares or Orthogonal Matching Pursuit?

TL;DR

This paper compares orthogonal least squares (OLS) and orthogonal matching pursuit (OMP) for sparse representation classification, demonstrating that OLS provides more accurate signal estimation and improves classification performance, especially with a kernelized variant.

Contribution

It introduces a classwise OLS-based classification method and its kernelized version, showing superior performance over traditional SRC and SVM on hyperspectral datasets.

Findings

OLS outperforms OMP in signal reconstruction accuracy.

Classwise OLS improves classification results.

Kernelized OLS handles nonlinearly separable data effectively.

Abstract

Spare representation of signals has received significant attention in recent years. Based on these developments, a sparse representation-based classification (SRC) has been proposed for a variety of classification and related tasks, including face recognition. Recently, a class dependent variant of SRC was proposed to overcome the limitations of SRC for remote sensing image classification. Traditionally, greedy pursuit based method such as orthogonal matching pursuit (OMP) are used for sparse coefficient recovery due to their simplicity as well as low time-complexity. However, orthogonal least square (OLS) has not yet been widely used in classifiers that exploit the sparse representation properties of data. Since OLS produces lower signal reconstruction error than OMP under similar conditions, we hypothesize that more accurate signal estimation will further improve the classification performance of classifiers that exploiting the sparsity of data. In this paper, we present a classification method based on OLS, which implements OLS in a classwise manner to perform the classification. We also develop and present its kernelized variant to handle nonlinearly separable data. Based on two real-world benchmarking hyperspectral datasets, we demonstrate that class dependent OLS based methods outperform several baseline methods including traditional SRC and the support vector machine classifier.