Weighted-Average Least Squares for Negative Binomial Regression

TL;DR

This paper extends weighted-average least squares (WALS) to negative binomial regression, demonstrating improved prediction over traditional methods in overdispersed count data with efficient computation.

Contribution

It introduces WALS for negative binomial models, combining Bayesian and frequentist approaches with a semiorthogonal transformation to handle covariate uncertainty efficiently.

Findings

WALS outperforms maximum likelihood in sparse data scenarios.

WALS is competitive with lasso in predictive accuracy.

WALS offers computational advantages over existing methods.

Abstract

Model averaging methods have become an increasingly popular tool for improving predictions and dealing with model uncertainty, especially in Bayesian settings. Recently, frequentist model averaging methods such as information theoretic and least squares model averaging have emerged. This work focuses on the issue of covariate uncertainty where managing the computational resources is key: The model space grows exponentially with the number of covariates such that averaged models must often be approximated. Weighted-average least squares (WALS), first introduced for (generalized) linear models in the econometric literature, combines Bayesian and frequentist aspects and additionally employs a semiorthogonal transformation of the regressors to reduce the computational burden. This paper extends WALS for generalized linear models to the negative binomial (NB) regression model for overdispersed count data. A simulation experiment and an empirical application using data on doctor visits were conducted to compare the predictive power of WALS for NB regression to traditional estimators. The results show that WALS for NB improves on the maximum likelihood estimator in sparse situations and is competitive with lasso while being computationally more efficient.