Divergent estimation error in portfolio optimization and in linear regression

TL;DR

This paper analyzes how estimation error in portfolio optimization and linear regression diverges at a critical ratio of variables to data points, revealing universal phase transition phenomena with broad implications across multiple fields.

Contribution

It identifies the divergence of estimation error as a phase transition phenomenon and extends the analysis to various high-dimensional regression problems beyond finance.

Findings

Estimation error diverges at a critical N/T ratio.

The divergence exhibits universality and critical phenomena.

Implications extend to machine learning, bioinformatics, and other fields.

Abstract

The problem of estimation error in portfolio optimization is discussed, in the limit where the portfolio size N and the sample size T go to infinity such that their ratio is fixed. The estimation error strongly depends on the ratio N/T and diverges for a critical value of this parameter. This divergence is the manifestation of an algorithmic phase transition, it is accompanied by a number of critical phenomena, and displays universality. As the structure of a large number of multidimensional regression and modelling problems is very similar to portfolio optimization, the scope of the above observations extends far beyond finance, and covers a large number of problems in operations research, machine learning, bioinformatics, medical science, economics, and technology.