Autocorrelated Optimize-via-Estimate: Predict-then-Optimize versus Finite-sample Optimal

TL;DR

This paper introduces A-OVE, a model for data-driven optimization under autocorrelated uncertainties, demonstrating its superior out-of-sample decision quality over traditional predict-then-optimize methods in portfolio management.

Contribution

The paper develops A-OVE, an out-of-sample optimal solution method for autocorrelated data, with a recursive computation of sufficient statistics, and compares it to existing machine learning benchmarks.

Findings

A-OVE achieves low regret compared to a perfect oracle.

Predict-then-optimize ML models can perform worse despite higher accuracy.

Performance of A-OVE is robust to small model mis-specification.

Abstract

Models that directly optimize for out-of-sample performance in the finite-sample regime have emerged as a promising alternative to traditional estimate-then-optimize approaches in data-driven optimization. In this work, we compare their performance in the context of autocorrelated uncertainties, specifically, under a Vector Autoregressive Moving Average VARMA(p,q) process. We propose an autocorrelated Optimize-via-Estimate (A-OVE) model that obtains an out-of-sample optimal solution as a function of sufficient statistics, and propose a recursive form for computing its sufficient statistics. We evaluate these models on a portfolio optimization problem with trading costs. A-OVE achieves low regret relative to a perfect information oracle, outperforming predict-then-optimize machine learning benchmarks. Notably, machine learning models with higher accuracy can have poorer decision quality, echoing the growing literature in data-driven optimization. Performance is retained under small mis-specification.