High-dimensionality effects in the Markowitz problem and other quadratic programs with linear constraints: Risk underestimation

TL;DR

This paper investigates how high-dimensionality affects quadratic programming solutions, especially in finance, revealing that risk is systematically underestimated due to estimation errors in mean and covariance, with implications for portfolio optimization.

Contribution

It provides a theoretical analysis of the impact of high-dimensional data on quadratic programs, particularly quantifying risk underestimation in the Markowitz problem.

Findings

Risk underestimation factor approximately 1 - p/n

Bias in linear functionals of empirical solutions

Separation of errors from mean and covariance estimation

Abstract

We first study the properties of solutions of quadratic programs with linear equality constraints whose parameters are estimated from data in the high-dimensional setting where p, the number of variables in the problem, is of the same order of magnitude as n, the number of observations used to estimate the parameters. The Markowitz problem in Finance is a subcase of our study. Assuming normality and independence of the observations we relate the efficient frontier computed empirically to the "true" efficient frontier. Our computations show that there is a separation of the errors induced by estimating the mean of the observations and estimating the covariance matrix. In particular, the price paid for estimating the covariance matrix is an underestimation of the variance by a factor roughly equal to 1-p/n. Therefore the risk of the optimal population solution is underestimated when we estimate it by solving a similar quadratic program with estimated parameters. We also characterize the statistical behavior of linear functionals of the empirical optimal vector and show that they are biased estimators of the corresponding population quantities.