DistPred: A Distribution-Free Probabilistic Inference Method for Regression and Forecasting

TL;DR

DistPred is a novel distribution-free probabilistic inference method for regression and forecasting that efficiently estimates response variable distributions with state-of-the-art accuracy and significantly improved computational speed.

Contribution

It introduces a differentiable loss based on proper scoring rules, enabling end-to-end training and single-pass sampling for distribution estimation.

Findings

Achieves state-of-the-art performance on multiple datasets.

Significantly faster inference, up to 180x compared to existing methods.

Reproducible experimental results available online.

Abstract

Traditional regression and prediction tasks often only provide deterministic point estimates. To estimate the distribution or uncertainty of the response variable, traditional methods either assume that the posterior distribution of samples follows a Gaussian process or require thousands of forward passes for sample generation. We propose a novel approach called DistPred for regression and forecasting tasks, which overcomes the limitations of existing methods while remaining simple and powerful. Specifically, we transform proper scoring rules that measure the discrepancy between the predicted distribution and the target distribution into a differentiable discrete form and use it as a loss function to train the model end-to-end. This allows the model to sample numerous samples in a single forward pass to estimate the potential distribution of the response variable. We have compared our method with several existing approaches on multiple datasets and achieved state-of-the-art performance. Additionally, our method significantly improves computational efficiency. For example, compared to state-of-the-art models, DistPred has a 180x faster inference speed Experimental results can be reproduced through https://github.com/Anoise/DistPred.