Action-State Dependent Dynamic Model Selection

TL;DR

This paper develops a reinforcement learning approach to dynamically select the best model based on current state information, optimizing performance in contexts like portfolio rebalancing with costs.

Contribution

It introduces a reinforcement learning algorithm for dynamic model selection that accounts for switching costs and demonstrates its effectiveness in empirical portfolio management.

Findings

The algorithm consistently estimates the optimal policy for model switching.

Empirical results show superior portfolio performance compared to static model selection.

The method effectively incorporates macroeconomic variables for decision-making.

Abstract

A model among many may only be best under certain states of the world. Switching from a model to another can also be costly. Finding a procedure to dynamically choose a model in these circumstances requires to solve a complex estimation procedure and a dynamic programming problem. A Reinforcement learning algorithm is used to approximate and estimate from the data the optimal solution to this dynamic programming problem. The algorithm is shown to consistently estimate the optimal policy that may choose different models based on a set of covariates. A typical example is the one of switching between different portfolio models under rebalancing costs, using macroeconomic information. Using a set of macroeconomic variables and price data, an empirical application to the aforementioned portfolio problem shows superior performance to choosing the best portfolio model with hindsight.