Regularized Ordinal Regression and the ordinalNet R Package

TL;DR

This paper introduces the ordinalNet R package, implementing a coordinate descent algorithm for regularized ordinal regression models with elastic net penalties, accommodating various data types and improving variable selection.

Contribution

It presents a novel algorithm and software package for regularized ordinal regression, extending flexibility to unordered data and non-ordinal models.

Findings

Efficient coordinate descent algorithm for ordinal regression.

Flexible model generalizations for unordered categorical data.

Implementation of the ordinalNet R package.

Abstract

Regularization techniques such as the lasso (Tibshirani 1996) and elastic net (Zou and Hastie 2005) can be used to improve regression model coefficient estimation and prediction accuracy, as well as to perform variable selection. Ordinal regression models are widely used in applications where the use of regularization could be beneficial; however, these models are not included in many popular software packages for regularized regression. We propose a coordinate descent algorithm to fit a broad class of ordinal regression models with an elastic net penalty. Furthermore, we demonstrate that each model in this class generalizes to a more flexible form, for instance to accommodate unordered categorical data. We introduce an elastic net penalty class that applies to both model forms. Additionally, this penalty can be used to shrink a non-ordinal model toward its ordinal counterpart. Finally, we introduce the R package ordinalNet, which implements the algorithm for this model class.