Bayesian Environment Invariant Regression

TL;DR

This paper introduces a Bayesian method for invariant regression across heterogeneous environments, explicitly modeling shared and environment-specific effects to improve robustness under distributional shifts.

Contribution

It proposes a Bayesian framework with a spike-and-slab prior to distinguish invariant from non-invariant effects, with theoretical guarantees and practical refinements.

Findings

The method achieves model selection consistency for invariant effects.

Posterior contraction is established for invariant coefficients.

Simulations and real data demonstrate robustness and efficiency.

Abstract

The availability of data from multiple heterogeneous environments has motivated methods that remain reliable under distributional shifts. When the joint distribution of response and predictors varies across environments, the response may still depend on a subset of predictors through an invariant mechanism. Existing methods typically assess candidate invariant sets through pooled stability criteria, treating environmental variation as nuisance. In this paper, we propose a Bayesian framework that explicitly separates a shared response mechanism from environment-specific or response-dependent associations, exploiting heterogeneity as evidence for structure learning. A competitive spike-and-slab prior is designed to force each predictor to compete between invariant and non-invariant spurious effects. Under a tractable working model, we establish invariant model selection consistency and posterior contraction for invariant coefficients. We further study the presence of irrelevant predictors, characterize posterior concentration on an equivalent invariant class, and introduce a post-selection refinement that consistently recovers the minimal invariant model. Simulations and a real application illustrate the robustness and finite-sample efficiency of the proposed method.