Do covariates explain why these groups differ? The choice of reference group can reverse conclusions in the Oaxaca-Blinder decomposition

TL;DR

This paper investigates how the choice of reference group in the Oaxaca--Blinder decomposition can lead to different substantive conclusions, highlighting the importance of reference selection in causal analysis.

Contribution

It provides the first systematic analysis showing that reference choice can significantly alter results, with proofs and empirical evidence of when discrepancies occur.

Findings

Up to half of the parameter space can yield different conclusions based on reference choice.

Discrepancies are rare in real data due to data-generating process biases.

The paper offers proofs and simulations demonstrating the impact of reference selection.

Abstract

Scientists often want to explain why an outcome is different in two groups. For instance, differences in patient mortality rates across two hospitals could be due to differences in the patients themselves (covariates) or differences in medical care (outcomes given covariates). The Oaxaca--Blinder decomposition (OBD) is a standard tool to tease apart these factors. It is well known that the OBD requires choosing one of the groups as a reference, and the numerical answer can vary with the reference. To the best of our knowledge, there has not been a systematic investigation into whether the choice of OBD reference can yield different substantive conclusions and how common this issue is. In the present paper, we give existence proofs in real and simulated data that the OBD references can yield substantively different conclusions and that these differences are not entirely driven by model misspecification or small data. We prove that substantively different conclusions occur in up to half of the parameter space, but find these discrepancies rare in the real-data analyses we study. We explain this empirical rarity by examining how realistic data-generating processes can be biased towards parameters that do not change conclusions under the OBD.