Efficient Solution of Portfolio Optimization Problems via Dimension Reduction and Sparsification

TL;DR

This paper introduces a novel approach combining machine learning-based dimension reduction and sparsification of the covariance matrix to efficiently solve large-scale portfolio optimization problems while maintaining performance.

Contribution

It proposes a new method that reduces covariance matrix size and increases sparsity, significantly improving computational efficiency in large portfolio optimizations.

Findings

Reduced runtime and iterations in optimization.

Predictions from linear programming outperform neural networks.

Maintained similar risk-return profiles with sparser matrices.

Abstract

The Markowitz mean-variance portfolio optimization model aims to balance expected return and risk when investing. However, there is a significant limitation when solving large portfolio optimization problems efficiently: the large and dense covariance matrix. Since portfolio performance can be potentially improved by considering a wider range of investments, it is imperative to be able to solve large portfolio optimization problems efficiently, typically in microseconds. We propose dimension reduction and increased sparsity as remedies for the covariance matrix. The size reduction is based on predictions from machine learning techniques and the solution to a linear programming problem. We find that using the efficient frontier from the linear formulation is much better at predicting the assets on the Markowitz efficient frontier, compared to the predictions from neural networks. Reducing the covariance matrix based on these predictions decreases both runtime and total iterations. We also present a technique to sparsify the covariance matrix such that it preserves positive semi-definiteness, which improves runtime per iteration. The methods we discuss all achieved similar portfolio expected risk and return as we would obtain from a full dense covariance matrix but with improved optimizer performance.