A Simulation Study to Compare Inferential Properties when Modelling Ordinal Outcomes: The Case for the (Plain but Robust) Proportional Odds Model

TL;DR

This study compares the inferential properties of various ordinal regression models through extensive simulations, highlighting the robustness and reliability of the proportional odds model for analyzing ordinal outcomes in social sciences.

Contribution

It provides a comprehensive simulation-based evaluation of common ordinal regression models, emphasizing the advantages of the proportional odds model over alternatives.

Findings

Proportional odds and linear models showed the most reliable hypothesis testing results.

Cumulative ordinal models exhibited large biases with high skewness and large parameters.

The proportional odds model is recommended for ordinal data analysis unless contraindicated.

Abstract

Ordinal measurements are common outcomes in studies within psychology, as well as in the social and behavioral sciences. Choosing an appropriate regression model for analysing such data poses a difficult task. This paper aims to facilitate modeling decisions for quantitative researchers by presenting the results of an extensive simulation study on the inferential properties of common ordinal regression models: the proportional odds model, the category-specific odds model, the location-shift model, the location-scale model, and the linear model, which incorrectly treats ordinal outcomes as metric. The simulations were conducted under different data generating processes based on each of the ordinal models and varying parameter configurations within each model class. We examined the bias of parameter estimates as well as type I error rates ($\alpha$-errors) and the power of statistical parameter testing procedures corresponding to the respective models. Our findings reveal several highlights. For parameter estimates, we observed that cumulative ordinal regression models exhibited large biases in cases of large parameter values and high skewness of the outcome distribution in the true data generation process. Regarding statistical hypothesis testing, the proportional odds model and the linear model showed the most reliable results. Due to its better fit and interpretability for ordinal outcomes, we recommend the use of the proportional odds model unless there are relevant contraindications.