Robust Locality-Aware Regression for Labeled Data Classification

TL;DR

This paper introduces Robust Locality-Aware Regression (RLAR), a novel feature extraction method that improves classification by adaptively learning margins and local data structures while being robust to outliers.

Contribution

RLAR jointly learns locality-aware graph structures and projection matrices with regularization, enhancing feature extraction and classification in high-dimensional data.

Findings

RLAR outperforms state-of-the-art methods on UCI datasets.

The model effectively handles outliers and prevents overfitting.

Feature selection and extraction are integrated through row sparsity.

Abstract

With the dramatic increase of dimensions in the data representation, extracting latent low-dimensional features becomes of the utmost importance for efficient classification. Aiming at the problems of unclear margin representation and difficulty in revealing the data manifold structure in most of the existing linear discriminant methods, we propose a new discriminant feature extraction framework, namely Robust Locality-Aware Regression (RLAR). In our model, we introduce a retargeted regression to perform the marginal representation learning adaptively instead of using the general average inter-class margin. Besides, we formulate a new strategy for enhancing the local intra-class compactness of the data manifold, which can achieve the joint learning of locality-aware graph structure and desirable projection matrix. To alleviate the disturbance of outliers and prevent overfitting, we measure the regression term and locality-aware term together with the regularization term by the L2,1 norm. Further, forcing the row sparsity on the projection matrix through the L2,1 norm achieves the cooperation of feature selection and feature extraction. Then, we derive an effective iterative algorithm for solving the proposed model. The experimental results over a range of UCI data sets and other benchmark databases demonstrate that the proposed RLAR outperforms some state-of-the-art approaches.