Introducing the b-value: combining unbiased and biased estimators from a sensitivity analysis perspective

TL;DR

This paper introduces the b-value, a new concept for combining biased and unbiased estimators in empirical research, providing a way to conduct valid inference under unknown bias and assess the robustness of conclusions.

Contribution

It proposes a novel sensitivity analysis framework with confidence intervals indexed by bias, and introduces the b-value to determine when combined estimators yield significant results.

Findings

Derived confidence intervals for combined estimators.

Characterized the bias threshold (b-value) for conclusion changes.

Recommended the soft-thresholding estimator for robust inference.

Abstract

In empirical research, when we have multiple estimators for the same parameter of interest, a central question arises: how do we combine unbiased but less precise estimators with biased but more precise ones to improve the inference? Under this setting, the point estimation problem has attracted considerable attention. In this paper, we focus on a less studied inference question: how can we conduct valid statistical inference in such settings with unknown bias? We propose a strategy to combine unbiased and biased estimators from a sensitivity analysis perspective. We derive a sequence of confidence intervals indexed by the magnitude of the bias, which enable researchers to assess how conclusions vary with the bias levels. Importantly, we introduce the notion of the b-value, a critical value of the unknown maximum relative bias at which combining estimators does not yield a significant result. We apply this strategy to three canonical combined estimators: the precision-weighted estimator, the pretest estimator, and the soft-thresholding estimator. For each estimator, we characterize the sequence of confidence intervals and determine the bias threshold at which the conclusion changes. Based on the theory, we recommend reporting the b-value based on the soft-thresholding estimator and its associated confidence intervals, which are robust to unknown bias and achieve the lowest worst-case risk among the alternatives.