Variable selection and basis learning for ordinal classification

TL;DR

This paper introduces a novel variable selection and basis learning method for high-dimensional ordinal classification, effectively identifying relevant variables and their order-concordance with responses, with strong theoretical guarantees.

Contribution

It extends sparse multiclass linear discriminant analysis to incorporate ordinal information, providing a high-dimensional consistent variable selection method with interpretability.

Findings

Method accurately separates ordinal from non-ordinal variables.

Consistent variable selection under high-dimensional asymptotics.

Real data analysis shows sparse, interpretable basis mostly with ordinal variables.

Abstract

We propose a method for variable selection and basis learning for high-dimensional classification with ordinal responses. The proposed method extends sparse multiclass linear discriminant analysis, with the aim of identifying not only the variables relevant to discrimination but also the variables that are order-concordant with the responses. For this purpose, we compute for each variable an ordinal weight, where larger weights are given to variables with ordered group-means, and penalize the variables with smaller weights more severely. A two-step construction for ordinal weights is developed, and we show that the ordinal weights correctly separate ordinal variables from non-ordinal variables with high probability. The resulting sparse ordinal basis learning method is shown to consistently select either the discriminant variables or the ordinal and discriminant variables, depending on the choice of a tunable parameter. Such asymptotic guarantees are given under a high-dimensional asymptotic regime where the dimension grows much faster than the sample size. We also discuss a two-step procedure of post-screening ordinal variables among the selected discriminant variables. Simulated and real data analyses confirm that the proposed basis learning provides sparse and interpretable basis, as it mostly consists of ordinal variables.