Automatic Construction and Natural-Language Description of Nonparametric Regression Models

TL;DR

This paper introduces an automatic system for constructing nonparametric regression models using Gaussian processes, which can describe data and generate detailed reports with natural language, achieving state-of-the-art extrapolation on real datasets.

Contribution

It presents a novel approach combining Gaussian processes with a compositional language for automatic model discovery and explanation in regression tasks.

Findings

Achieves state-of-the-art extrapolation performance on 13 real datasets.

Automatically describes functions using high-level properties like smoothness and periodicity.

Generates detailed reports with figures and natural-language summaries.

Abstract

This paper presents the beginnings of an automatic statistician, focusing on regression problems. Our system explores an open-ended space of statistical models to discover a good explanation of a data set, and then produces a detailed report with figures and natural-language text. Our approach treats unknown regression functions nonparametrically using Gaussian processes, which has two important consequences. First, Gaussian processes can model functions in terms of high-level properties (e.g. smoothness, trends, periodicity, changepoints). Taken together with the compositional structure of our language of models this allows us to automatically describe functions in simple terms. Second, the use of flexible nonparametric models and a rich language for composing them in an open-ended manner also results in state-of-the-art extrapolation performance evaluated over 13 real time series data sets from various domains.