Distribution Regression

TL;DR

This paper introduces distribution regression, a flexible linear regression method using nonparametric techniques that better handles asymmetrical heavy-tailed and multimodal error distributions, with strong theoretical and empirical support.

Contribution

It proposes a novel distribution regression approach that extends linear regression to broad error distributions, with proven consistency, asymptotic normality, and oracle properties.

Findings

Outperforms mean and quantile regression in various settings

Estimator is $\

, and possesses oracle properties with diverging parameters,

Abstract

Linear regression is a fundamental and popular statistical method. There are various kinds of linear regression, such as mean regression and quantile regression. In this paper, we propose a new one called distribution regression, which allows broad-spectrum of the error distribution in the linear regression. Our method uses nonparametric technique to estimate regression parameters. Our studies indicate that our method provides a better alternative than mean regression and quantile regression under many settings, particularly for asymmetrical heavy-tailed distribution or multimodal distribution of the error term. Under some regular conditions, our estimator is $\sqrt n$-consistent and possesses the asymptotically normal distribution. The proof of the asymptotic normality of our estimator is very challenging because our nonparametric likelihood function cannot be transformed into sum of independent and identically distributed random variables. Furthermore, penalized likelihood estimator is proposed and enjoys the so-called oracle property with diverging number of parameters. Numerical studies also demonstrate the effectiveness and the flexibility of the proposed method.