A Meta Reinforcement Learning Approach to Goals-Based Wealth Management

TL;DR

This paper introduces MetaRL, a meta reinforcement learning method pre-trained on numerous wealth management problems, enabling near-instantaneous, high-utility investment strategies adaptable to different market regimes.

Contribution

It develops a pre-trained MetaRL model that generalizes across goals-based wealth management problems, reducing computation and improving robustness compared to traditional methods.

Findings

MetaRL achieves 97.8% of optimal expected utility.

The approach is robust to market regime changes.

MetaRL can handle larger state spaces where Dynamic Programming fails.

Abstract

Applying concepts related to zero-shot meta-learning and pre-training of foundation models, we develop a meta reinforcement learning approach (denoted MetaRL) that is pre-trained on thousands of goals-based wealth management (GBWM) problems. Each GBWM problem involves a multiple year scenario over which the investor looks to optimally choose an investment portfolio each year and choose to fulfill all, some, or none of the different financial goals that arise each year. These choices seek to maximize the expected total investor utility obtained from the fulfilled financial goals. By eliminating separate training and optimization for each new investor problem, the MetaRL model in inference mode produces near-optimal dynamic investment portfolio and goal-fulfilling strategies for a new GBWM problem within a few hundredths of a second. This delivers expected utilities that are, on average, 97.8% of the optimal expected utilities (determined via Dynamic Programming). These results are remarkably robust to capital market regime changes, even when training uses only one capital market regime. Further, the MetaRL approach can enable solving problems with larger state spaces where Dynamic Programming becomes computationally infeasible.