Neural Spline Search for Quantile Probabilistic Modeling

TL;DR

This paper introduces Neural Spline Search (NSS), a non-parametric, flexible method for accurately modeling data distributions and quantile functions without assuming specific parametric forms, improving quantile regression tasks.

Contribution

The paper presents NSS, a novel non-parametric approach using monotonic spline transformations guided by symbolic operators for expressive distribution modeling.

Findings

NSS outperforms previous methods on synthetic data.

NSS achieves better results on real-world regression tasks.

NSS improves quantile estimation accuracy.

Abstract

Accurate estimation of output quantiles is crucial in many use cases, where it is desired to model the range of possibility. Modeling target distribution at arbitrary quantile levels and at arbitrary input attribute levels are important to offer a comprehensive picture of the data, and requires the quantile function to be expressive enough. The quantile function describing the target distribution using quantile levels is critical for quantile regression. Although various parametric forms for the distributions (that the quantile function specifies) can be adopted, an everlasting problem is selecting the most appropriate one that can properly approximate the data distributions. In this paper, we propose a non-parametric and data-driven approach, Neural Spline Search (NSS), to represent the observed data distribution without parametric assumptions. NSS is flexible and expressive for modeling data distributions by transforming the inputs with a series of monotonic spline regressions guided by symbolic operators. We demonstrate that NSS outperforms previous methods on synthetic, real-world regression and time-series forecasting tasks.