Empirical Bayes for Data Integration

TL;DR

This paper explores empirical Bayes methods for data integration in transfer learning, demonstrating their advantages in variable selection consistency, convergence speed, and practical performance improvements over full Bayesian approaches.

Contribution

It develops a computational framework for empirical Bayes in data integration, showing its theoretical benefits and practical effectiveness in high-dimensional settings.

Findings

Empirical Bayes achieves consistent variable selection under weaker conditions.

It attains faster convergence rates than full Bayes.

Data integration with empirical Bayes provides meaningful practical improvements.

Abstract

We discuss the use of empirical Bayes for data integration, in the sense of transfer learning. Our main interest is in settings where one wishes to learn structure (e.g. feature selection) and one only has access to incomplete data from previous studies, such as summaries, estimates or lists of relevant features. We discuss differences between full Bayes and empirical Bayes, and develop a computational framework for the latter. We discuss how empirical Bayes attains consistent variable selection under weaker conditions (sparsity and betamin assumptions) than full Bayes and other standard criteria do, and how it attains faster convergence rates. Our high-dimensional regression examples show that fully Bayesian inference enjoys excellent properties, and that data integration with empirical Bayes can offer moderate yet meaningful improvements in practice.