Empirical Bayes Model Averaging with Influential Observations: Tuning Zellner's g Prior for Predictive Robustness

TL;DR

This paper investigates how to improve Bayesian model averaging in linear regression when influential observations cause model misfit, by tuning Zellner's g prior to enhance predictive robustness using an empirical Bayes approach.

Contribution

It introduces a method for tuning Zellner's g prior in BMA to mitigate the effects of influential observations and improve predictive accuracy.

Findings

Guidelines for tuning Zellner's g prior based on theoretical properties.

Demonstrates improved predictive robustness with the proposed tuning method.

Provides empirical evidence of reduced impact of influential observations.

Abstract

The behavior of Bayesian model averaging (BMA) for the normal linear regression model in the presence of influential observations that contribute to model misfit is investigated. Remedies to attenuate the potential negative impacts of such observations on inference and prediction are proposed. The methodology is motivated by the view that well-behaved residuals and good predictive performance often go hand-in-hand. Focus is placed on regression models that use variants on Zellner's g prior. Studying the impact of various forms of model misfit on BMA predictions in simple situations points to prescriptive guidelines for "tuning" Zellner's g prior to obtain optimal predictions. The tuning of the prior distribution is obtained by considering theoretical properties that should be enjoyed by the optimal fits of the various models in the BMA ensemble. The methodology can be thought of as an "empirical Bayes" approach to modeling, as the data help to inform the specification of the prior in an attempt to attenuate the negative impact of influential cases.