The Median Probability Model and Correlated Variables

TL;DR

This paper investigates the properties and robustness of the median probability model (MPM) in correlated variable settings, extending its applicability to broader priors and emphasizing the role of prior probabilities.

Contribution

It characterizes when MPM remains effective under correlated designs, generalizes MPM to continuous spike-and-slab priors, and highlights the importance of prior model probabilities.

Findings

MPM is effective in orthogonal and nested correlated designs.

Generalizations of MPM to broader priors are proposed.

Support for g-priors over independent priors is provided.

Abstract

The median probability model (MPM) Barbieri and Berger (2004) is defined as the model consisting of those variables whose marginal posterior probability of inclusion is at least 0.5. The MPM rule yields the best single model for prediction in orthogonal and nested correlated designs. This result was originally conceived under a specific class of priors, such as the point mass mixtures of non-informative and g-type priors. The MPM rule, however, has become so very popular that it is now being deployed for a wider variety of priors and under correlated designs, where the properties of MPM are not yet completely understood. The main thrust of this work is to shed light on properties of MPM in these contexts by (a) characterizing situations when MPM is still safe under correlated designs, (b) providing significant generalizations of MPM to a broader class of priors (such as continuous spike-and-slab priors). We also provide new supporting evidence for the suitability of g-priors, as opposed to independent product priors, using new predictive matching arguments. Furthermore, we emphasize the importance of prior model probabilities and highlight the merits of non-uniform prior probability assignments using the notion of model aggregates.