Remember to correct the bias when using deep learning for regression!

TL;DR

This paper highlights the importance of bias correction in deep learning regression models to prevent systematic errors from accumulating, proposing a postprocessing adjustment as an effective solution.

Contribution

It introduces a simple bias correction method as a default postprocessing step for deep learning regression models to improve accuracy.

Findings

Bias correction reduces systematic errors in regression

Postprocessing bias adjustment improves aggregated performance

Experiments demonstrate effectiveness of the method

Abstract

When training deep learning models for least-squares regression, we cannot expect that the training error residuals of the final model, selected after a fixed training time or based on performance on a hold-out data set, sum to zero. This can introduce a systematic error that accumulates if we are interested in the total aggregated performance over many data points. We suggest to adjust the bias of the machine learning model after training as a default postprocessing step, which efficiently solves the problem. The severeness of the error accumulation and the effectiveness of the bias correction is demonstrated in exemplary experiments.