High-Dimensional Expected Shortfall Regression

TL;DR

This paper introduces a novel lasso-penalized expected shortfall regression method for high-dimensional data, enabling robust analysis of tail-related effects and providing valid inference tools, with applications demonstrated in health disparity research.

Contribution

It develops a new high-dimensional expected shortfall regression framework with theoretical error bounds and inference procedures, addressing challenges in tail analysis with sparse models.

Findings

Non-asymptotic error bounds established

Asymptotic normality of debiased estimator proven

Method demonstrated effective in health disparity data

Abstract

Expected shortfall is defined as the average over the tail below (or above) a certain quantile of a probability distribution. Expected shortfall regression provides powerful tools for learning the relationship between a response variable and a set of covariates while exploring the heterogeneous effects of the covariates. In the health disparity research, for example, the lower/upper tail of the conditional distribution of a health-related outcome, given high-dimensional covariates, is often of importance. Under sparse models, we propose the lasso-penalized expected shortfall regression and establish non-asymptotic error bounds, depending explicitly on the sample size, dimension, and sparsity, for the proposed estimator. To perform statistical inference on a covariate of interest, we propose a debiased estimator and establish its asymptotic normality, from which asymptotically valid tests can be constructed. We illustrate the finite sample performance of the proposed method through numerical studies and a data application on health disparity.