The risk of bias in denoising methods

TL;DR

Denoising methods in data analysis can reduce measurement noise but may introduce bias, potentially leading to incorrect scientific conclusions, and researchers should evaluate bias alongside variance reduction.

Contribution

This paper highlights the risk of bias in denoising techniques, providing simulations that demonstrate how bias can occur even when variance is effectively reduced.

Findings

Different denoising methods can have similar variance reduction but vary in bias.

Methods that recover ground truth do not necessarily avoid bias.

Bias can occur even with knowledge of the signal's properties.

Abstract

Experimental datasets are growing rapidly in size, scope, and detail, but the value of these datasets is limited by unwanted measurement noise. It is therefore tempting to apply analysis techniques that attempt to reduce noise and enhance signals of interest. In this paper, we draw attention to the possibility that denoising methods may introduce bias and lead to incorrect scientific inferences. To present our case, we first review the basic statistical concepts of bias and variance. Denoising techniques typically reduce variance observed across repeated measurements, but this can come at the expense of introducing bias to the average expected outcome. We then conduct three simple simulations that provide concrete examples of how bias may manifest in everyday situations. These simulations reveal several findings that may be surprising and counterintuitive: (i) different methods can be equally effective at reducing variance but some incur bias while others do not, (ii) identifying methods that better recover ground truth does not guarantee the absence of bias, (iii) bias can arise even if one has specific knowledge of properties of the signal of interest. We suggest that researchers should consider and possibly quantify bias before deploying denoising methods on important research data.