An Empirical Bayes Approach to Regularization Using Previously Published Models

TL;DR

This paper introduces an empirical Bayes regularization method that leverages previously published models to improve coefficient estimation in predictive modeling, offering a more research-informed shrinkage approach.

Contribution

It presents a novel empirical Bayes technique that uses existing model predictions for regularization, differing from traditional zero-centered shrinkage methods.

Findings

Effective in SNP dysfunction prediction

Improves preoperative serum creatinine change prediction

Outperforms traditional regularization methods

Abstract

This manuscript proposes a novel empirical Bayes technique for regularizing regression coefficients in predictive models. When predictions from a previously published model are available, this empirical Bayes method provides a natural mathematical framework for shrinking coefficients toward the estimates implied by the body of existing research rather than the shrinkage toward zero provided by traditional L1 and L2 penalization schemes. The method is applied to two different prediction problems. The first involves the construction of a model for predicting whether a single nucleotide polymorphism (SNP) of the KCNQ1 gene will result in dysfunction of the corresponding voltage gated ion channel. The second involves the prediction of preoperative serum creatinine change in patients undergoing cardiac surgery.