Structured Mixture of Continuation-ratio Logits Models for Ordinal Regression

TL;DR

This paper introduces a flexible Bayesian ordinal regression model using structured mixtures of multinomial distributions with continuation-ratio logits, enabling non-linear covariate effects and efficient computation.

Contribution

It develops a novel nonparametric Bayesian approach with a structured mixture model and Pólya-Gamma augmentation, improving flexibility and computational efficiency in ordinal regression.

Findings

Model achieves flexible ordinal regression relationships.

Efficient posterior simulation via Pólya-Gamma data augmentation.

Demonstrated effectiveness on synthetic and real data.

Abstract

We develop a nonparametric Bayesian modeling approach to ordinal regression based on priors placed directly on the discrete distribution of the ordinal responses. The prior probability models are built from a structured mixture of multinomial distributions. We leverage a continuation-ratio logits representation to formulate the mixture kernel, with mixture weights defined through the logit stick-breaking process that incorporates the covariates through a linear function. The implied regression functions for the response probabilities can be expressed as weighted sums of parametric regression functions, with covariate-dependent weights. Thus, the modeling approach achieves flexible ordinal regression relationships, avoiding linearity or additivity assumptions in the covariate effects. Model flexibility is formally explored through the Kullback-Leibler support of the prior probability model. A key model feature is that the parameters for both the mixture kernel and the mixture weights can be associated with a continuation-ratio logits regression structure. Hence, an efficient and relatively easy to implement posterior simulation method can be designed, using P\'olya-Gamma data augmentation. Moreover, the model is built from a conditional independence structure for category-specific parameters, which results in additional computational efficiency gains through partial parallel sampling. In addition to the general mixture structure, we study simplified model versions that incorporate covariate dependence only in the mixture kernel parameters or only in the mixture weights. For all proposed models, we discuss approaches to prior specification and develop Markov chain Monte Carlo methods for posterior simulation. The methodology is illustrated with several synthetic and real data examples.