Remarks on Loss Function of Threshold Method for Ordinal Regression Problem

TL;DR

This paper analyzes how data distribution and learning procedures affect the performance of threshold methods in ordinal regression, revealing limitations when distributions are non-unimodal or when certain loss functions are used.

Contribution

It provides theoretical insights and numerical evidence on the influence of data distribution and learning procedures on threshold method performance in ordinal regression.

Findings

Threshold methods may perform poorly with non-unimodal distributions.

Learned 1DT values tend to concentrate at few points under certain loss functions.

Performance is affected by the shape of the conditional probability distribution.

Abstract

Threshold methods are popular for ordinal regression problems, which are classification problems for data with a natural ordinal relation. They learn a one-dimensional transformation (1DT) of observations of the explanatory variable, and then assign label predictions to the observations by thresholding their 1DT values. In this paper, we study the influence of the underlying data distribution and of the learning procedure of the 1DT on the classification performance of the threshold method via theoretical considerations and numerical experiments. Consequently, for example, we found that threshold methods based on typical learning procedures may perform poorly when the probability distribution of the target variable conditioned on an observation of the explanatory variable tends to be non-unimodal. Another instance of our findings is that learned 1DT values are concentrated at a few points under the learning procedure based on a piecewise-linear loss function, which can make difficult to classify data well.