Discriminative Ridge Machine: A Classifier for High-Dimensional Data or Imbalanced Data

TL;DR

This paper introduces the Discriminative Regression Machine (DRM), a novel classifier for high-dimensional and imbalanced data that extends traditional regression models by explicitly incorporating discriminative information, achieving superior performance.

Contribution

The paper proposes DRM, a new discriminative regression model with closed-form solutions and iterative algorithms, applicable to various data types including high-dimensional and imbalanced datasets.

Findings

DRM outperforms state-of-the-art classifiers in experiments.

The approach effectively handles high-dimensional and imbalanced data.

Algorithms improve efficiency and scalability for real-world applications.

Abstract

We introduce a discriminative regression approach to supervised classification in this paper. It estimates a representation model while accounting for discriminativeness between classes, thereby enabling accurate derivation of categorical information. This new type of regression models extends existing models such as ridge, lasso, and group lasso through explicitly incorporating discriminative information. As a special case we focus on a quadratic model that admits a closed-form analytical solution. The corresponding classifier is called discriminative regression machine (DRM). Three iterative algorithms are further established for the DRM to enhance the efficiency and scalability for real applications. Our approach and the algorithms are applicable to general types of data including images, high-dimensional data, and imbalanced data. We compare the DRM with currently state-of-the-art classifiers. Our extensive experimental results show superior performance of the DRM and confirm the effectiveness of the proposed approach.