Bayesian Multinomial Logistic Regression for Numerous Categories

TL;DR

This paper introduces a gamma-augmentation method for Bayesian multinomial logistic regression that significantly improves computational efficiency when handling many outcome categories, enabling faster sampling and inference.

Contribution

It adapts a gamma-augmentation strategy to decouple category-specific updates, enhancing scalability for models with numerous classes.

Findings

Gamma-augmentation accelerates sampling in high-category scenarios.

The method outperforms standard competitors in effective sample size and rate.

Performance depends on dimension and class imbalance regimes.

Abstract

Bayesian multinomial logistic regression provides a principled, interpretable approach to multiclass classification, but posterior sampling becomes increasingly expensive as the model dimension grows. Prior work has studied scalability in the number of subjects and covariates; in contrast, this paper focuses on how computation changes as the number of outcome categories increases. To improve scalability in settings with numerous categories, we adapt a gamma-augmentation strategy to decouple category-specific coefficient updates, so that each category's coefficients can be updated conditional on a single auxiliary variable per subject, rather than on the full set of other categories' coefficients. Because the resulting coefficient conditionals are non-conjugate, we couple this augmentation with either adaptive Metropolis-Hastings or elliptical slice sampling. Through simulation and a real-data example, we compare effective sample size and effective sampling rate across several standard competitors. We find that the best-performing sampler depends on the dimension and imbalance regime, and that the proposed augmentation provides substantial speedups in scenarios with numerous categories.