Diversity and Sparsity: A New Perspective on Index Tracking

TL;DR

This paper introduces a novel index tracking method that jointly optimizes diversity and sparsity, leading to more accurate and risk-aware portfolios, validated through extensive 15-year backtesting.

Contribution

It presents the first framework explicitly combining diversity and sparsity in index tracking, using data-driven asset similarity to improve performance.

Findings

Outperforms existing methods in accuracy and risk reduction

Efficiently optimizes both diversity and sparsity constraints

Demonstrates superior out-of-sample performance over 15 years

Abstract

We address the problem of partial index tracking, replicating a benchmark index using a small number of assets. Accurate tracking with a sparse portfolio is extensively studied as a classic finance problem. However in practice, a tracking portfolio must also be diverse in order to minimise risk -- a requirement which has only been dealt with by ad-hoc methods before. We introduce the first index tracking method that explicitly optimises both diversity and sparsity in a single joint framework. Diversity is realised by a regulariser based on pairwise similarity of assets, and we demonstrate that learning similarity from data can outperform some existing heuristics. Finally, we show that the way we model diversity leads to an easy solution for sparsity, allowing both constraints to be optimised easily and efficiently. we run out-of-sample backtesting for a long interval of 15 years (2003 -- 2018), and the results demonstrate the superiority of the proposed algorithm.