A supervised discriminant data representation: application to pattern classification

TL;DR

This paper introduces a hybrid linear feature extraction method for supervised multi-class classification, combining advantages of recent discriminant analysis techniques to improve data representation and classification accuracy.

Contribution

It proposes a unifying criterion that integrates RSLDA and ICS_DLSR, utilizing sparsity-promoting techniques and an iterative optimization scheme for enhanced data representation.

Findings

Outperforms competing methods on face, object, and digit datasets

Effectively selects features that best represent data

Preserves class-specific sample structures

Abstract

The performance of machine learning and pattern recognition algorithms generally depends on data representation. That is why, much of the current effort in performing machine learning algorithms goes into the design of preprocessing frameworks and data transformations able to support effective machine learning. The method proposed in this work consists of a hybrid linear feature extraction scheme to be used in supervised multi-class classification problems. Inspired by two recent linear discriminant methods: robust sparse linear discriminant analysis (RSLDA) and inter-class sparsitybased discriminative least square regression (ICS_DLSR), we propose a unifying criterion that is able to retain the advantages of these two powerful methods. The resulting transformation relies on sparsity-promoting techniques both to select the features that most accurately represent the data and to preserve the row-sparsity consistency property of samples from the same class. The linear transformation and the orthogonal matrix are estimated using an iterative alternating minimization scheme based on steepest descent gradient method and different initialization schemes. The proposed framework is generic in the sense that it allows the combination and tuning of other linear discriminant embedding methods. According to the experiments conducted on several datasets including faces, objects, and digits, the proposed method was able to outperform competing methods in most cases.