Optimal pooling and distributed inference for the tail index and extreme quantiles

TL;DR

This paper develops optimal pooling strategies for tail index and extreme quantile estimation from heavy-tailed data, providing asymptotic theory, optimal weights, and demonstrating effectiveness in distributed inference and real data applications.

Contribution

It introduces weighted pooled estimators with optimal weights for tail analysis, extending to heterogeneous, dependent, and covariate-influenced data, with theoretical guarantees and practical validation.

Findings

Optimal weights minimize asymptotic variance and MSE.

Distributed estimators are asymptotically equivalent to ideal benchmarks.

Pooled estimators perform nearly as well as traditional estimators in simulations.

Abstract

This paper investigates pooling strategies for tail index and extreme quantile estimation from heavy-tailed data. To fully exploit the information contained in several samples, we present general weighted pooled Hill estimators of the tail index and weighted pooled Weissman estimators of extreme quantiles calculated through a nonstandard geometric averaging scheme. We develop their large-sample asymptotic theory across a fixed number of samples, covering the general framework of heterogeneous sample sizes with different and asymptotically dependent distributions. Our results include optimal choices of pooling weights based on asymptotic variance and MSE minimization. In the important application of distributed inference, we prove that the variance-optimal distributed estimators are asymptotically equivalent to the benchmark Hill and Weissman estimators based on the unfeasible combination of subsamples, while the AMSE-optimal distributed estimators enjoy a smaller AMSE than the benchmarks in the case of large bias. We consider additional scenarios where the number of subsamples grows with the total sample size and effective subsample sizes can be low. We extend our methodology to handle serial dependence and the presence of covariates. Simulations confirm that our pooled estimators perform virtually as well as the benchmark estimators. Two applications to real weather and insurance data are showcased.