Information Borrowing in Regression Models

TL;DR

This paper introduces a novel method for decomposing regression estimates into data-driven weights, called borrowing factors, applicable to both linear and Bayesian hierarchical models, to better understand data influence and model assumptions.

Contribution

It extends the concept of the hat matrix to Bayesian hierarchical models, providing a unified framework for quantifying data borrowing and shrinkage effects in regression analysis.

Findings

Borrowing factors generalize shrinkage and information borrowing across regression models.

Metrics derived from borrowing factors effectively summarize data influence and model behavior.

The method enhances understanding of data imbalance and influential points in regression models.

Abstract

Model development often takes data structure, subject matter considerations, model assumptions, and goodness of fit into consideration. To diagnose issues with any of these factors, it can be helpful to understand regression model estimates at a more granular level. We propose a new method for decomposing point estimates from a regression model via weights placed on data clusters. The weights are informed only by the model specification and data availability and thus can be used to explicitly link the effects of data imbalance and model assumptions to actual model estimates. The weight matrix has been understood in linear models as the hat matrix in the existing literature. We extend it to Bayesian hierarchical regression models that incorporate prior information and complicated dependence structures through the covariance among random effects. We show that the model weights, which we call borrowing factors, generalize shrinkage and information borrowing to all regression models. In contrast, the focus of the hat matrix has been mainly on the diagonal elements indicating the amount of leverage. We also provide metrics that summarize the borrowing factors and are practically useful. We present the theoretical properties of the borrowing factors and associated metrics and demonstrate their usage in two examples. By explicitly quantifying borrowing and shrinkage, researchers can better incorporate domain knowledge and evaluate model performance and the impacts of data properties such as data imbalance or influential points.