Why Machine Learning Models Systematically Underestimate Extreme Values II: How to Fix It with LatentNN

TL;DR

This paper introduces LatentNN, a neural network method that corrects attenuation bias caused by measurement errors, improving inference accuracy in low signal-to-noise astronomical data.

Contribution

It extends the latent variable approach to neural networks, providing a joint optimization framework to reduce bias in noisy data regimes.

Findings

LatentNN reduces attenuation bias across various noise levels.

Standard neural networks exhibit significant bias in low SNR conditions.

Code for LatentNN is available at the provided GitHub URL.

Abstract

Attenuation bias -- the systematic underestimation of regression coefficients due to measurement errors in input variables -- affects astronomical data-driven models. For linear regression, this problem was solved by treating the true input values as latent variables to be estimated alongside model parameters. In this paper, we show that neural networks suffer from the same attenuation bias and that the latent variable solution generalizes directly to neural networks. We introduce LatentNN, a method that jointly optimizes network parameters and latent input values by maximizing the joint likelihood of observing both inputs and outputs. We demonstrate the correction on one-dimensional regression, multivariate inputs with correlated features, and stellar spectroscopy applications. LatentNN reduces attenuation bias across a range of signal-to-noise ratios where standard neural networks show large bias. This provides a framework for improved neural network inference in the low signal-to-noise regime characteristic of astronomical data. This bias correction is most effective when measurement errors are less than roughly half the intrinsic data range; in the regime of very low signal-to-noise and few informative features. Code is available at https://github.com/tingyuansen/LatentNN.