Mean and median bias reduction in generalized linear models

TL;DR

This paper introduces a unified framework for mean and median bias reduction in generalized linear models, improving estimation accuracy and inference, especially in complex models with potential infinite estimates.

Contribution

It develops a comprehensive approach using adjusted score equations and a unifying algorithm, extending bias reduction techniques to new contexts like multinomial logistic regression.

Findings

Bias-reduced estimates avoid infinite estimates in complex models

The framework unifies mean and median bias reduction methods

Estimates enable valid inference in high-dimensional settings

Abstract

This paper presents an integrated framework for estimation and inference from generalized linear models using adjusted score equations that result in mean and median bias reduction. The framework unifies theoretical and methodological aspects of past research on mean bias reduction and accommodates, in a natural way, new advances on median bias reduction. General expressions for the adjusted score functions are derived in terms of quantities that are readily available in standard software for fitting generalized linear models. The resulting estimating equations are solved using a unifying quasi-Fisher scoring algorithm that is shown to be equivalent to iteratively re-weighted least squares with appropriately adjusted working variates. Formal links between the iterations for mean and median bias reduction are established. Core model invariance properties are used to develop a novel mixed adjustment strategy when the estimation of a dispersion parameter is necessary. It is also shown how median bias reduction in multinomial logistic regression can be done using the equivalent Poisson log-linear model. The estimates coming out from mean and median bias reduction are found to overcome practical issues related to infinite estimates that can occur with positive probability in generalized linear models with multinomial or discrete responses, and can result in valid inferences even in the presence of a high-dimensional nuisance parameter