Neural Regression For Scale-Varying Targets

TL;DR

This paper introduces autoregressive regression, a new training method for deep learning models that effectively handles scale-varying real-valued targets by modeling their distribution autoregressively.

Contribution

The paper proposes a novel autoregressive approach to regression that overcomes scale sensitivity issues and improves modeling of real-valued targets.

Findings

Autoregressive regression effectively handles targets with different scales.

It provides high-fidelity distribution modeling for real-valued targets.

The method is computationally efficient compared to histogram loss.

Abstract

In this work, we demonstrate that a major limitation of regression using a mean-squared error loss is its sensitivity to the scale of its targets. This makes learning settings consisting of target's whose values take on varying scales challenging. A recently-proposed alternative loss function, known as histogram loss, avoids this issue. However, its computational cost grows linearly with the number of buckets in the histogram, which renders prediction with real-valued targets intractable. To address this issue, we propose a novel approach to training deep learning models on real-valued regression targets, autoregressive regression, which learns a high-fidelity distribution by utilizing an autoregressive target decomposition. We demonstrate that this training objective allows us to solve regression tasks involving targets with different scales.