Unbiased Shrinkage Estimation

TL;DR

This paper introduces a method for unbiased shrinkage estimation in linear regression that reduces variance without bias by leveraging additional information about covariate distributions, outperforming standard estimators.

Contribution

It demonstrates that in certain linear models, a James-Stein type shrinkage estimator can dominate the least-squares estimator for treatment effect estimation, even among unbiased estimators.

Findings

Shrinkage estimator reduces variance compared to least squares.

Dominates least-squares estimator in squared-error loss for treatment effect.

Applicable with at least three control variables and exogenous treatment.

Abstract

Shrinkage estimation usually reduces variance at the cost of bias. But when we care only about some parameters of a model, I show that we can reduce variance without incurring bias if we have additional information about the distribution of covariates. In a linear regression model with homoscedastic Normal noise, I consider shrinkage estimation of the nuisance parameters associated with control variables. For at least three control variables and exogenous treatment, I establish that the standard least-squares estimator is dominated with respect to squared-error loss in the treatment effect even among unbiased estimators and even when the target parameter is low-dimensional. I construct the dominating estimator by a variant of James-Stein shrinkage in a high-dimensional Normal-means problem. It can be interpreted as an invariant generalized Bayes estimator with an uninformative (improper) Jeffreys prior in the target parameter.