Scaling Conditional Autoencoders for Portfolio Optimization via Uncertainty-Aware Factor Selection

TL;DR

This paper introduces a scalable conditional autoencoder framework with an uncertainty-based factor selection method that improves portfolio optimization performance by focusing on the most predictable factors and combining multiple models for better risk-adjusted returns.

Contribution

It presents a novel scalable framework coupling high-dimensional CAEs with an uncertainty-aware factor selection process for enhanced portfolio optimization.

Findings

Risk-adjusted performance improves with factor pruning.

Ensemble models outperform individual models.

Significant gains in Sharpe, Sortino, and Omega ratios.

Abstract

Conditional Autoencoders (CAEs) offer a flexible, interpretable approach for estimating latent asset-pricing factors from firm characteristics. However, existing studies usually limit the latent factor dimension to around K=5 due to concerns that larger K can degrade performance. To overcome this challenge, we propose a scalable framework that couples a high-dimensional CAE with an uncertainty-aware factor selection procedure. We employ three models for quantile prediction: zero-shot Chronos, a pretrained time-series foundation model (ZS-Chronos), gradient-boosted quantile regression trees using XGBoost and RAPIDS (Q-Boost), and an I.I.D bootstrap-based sample mean model (IID-BS). For each model, we rank factors by forecast uncertainty and retain the top-k most predictable factors for portfolio construction, where k denotes the selected subset of factors. This pruning strategy delivers substantial gains in risk-adjusted performance across all forecasting models. Furthermore, due to each model's uncorrelated predictions, a performance-weighted ensemble consistently outperforms individual models with higher Sharpe, Sortino, and Omega ratios.