Reinforcement Learning for Monetary Policy Under Macroeconomic Uncertainty: Analyzing Tabular and Function Approximation Methods

TL;DR

This paper evaluates various reinforcement learning methods for central bank monetary policy, finding simple tabular Q-learning surprisingly outperforms more complex algorithms and traditional rules in a macroeconomic setting.

Contribution

It compares multiple RL approaches for monetary policy, revealing that simple tabular Q-learning can outperform advanced methods and traditional rules in this context.

Findings

Tabular Q-learning achieved the highest mean return.

Complex RL methods did not outperform simple Q-learning.

Traditional policy rules were less effective than basic RL algorithms.

Abstract

We study how a central bank should dynamically set short-term nominal interest rates to stabilize inflation and unemployment when macroeconomic relationships are uncertain and time-varying. We model monetary policy as a sequential decision-making problem where the central bank observes macroeconomic conditions quarterly and chooses interest rate adjustments. Using publicly accessible historical Federal Reserve Economic Data (FRED), we construct a linear-Gaussian transition model and implement a discrete-action Markov Decision Process with a quadratic loss reward function. We chose to compare nine different reinforcement learning style approaches against Taylor Rule and naive baselines, including tabular Q-learning variants, SARSA, Actor-Critic, Deep Q-Networks, Bayesian Q-learning with uncertainty quantification, and POMDP formulations with partial observability. Notably, despite its simplicity, standard tabular Q-learning achieved the best performance (-615.13 +- 309.58 mean return), outperforming both enhanced RL methods and traditional policy rules. Our results suggest that while sophisticated RL techniques show promise for monetary policy applications, simpler approaches may be more robust in this domain, highlighting important challenges in applying modern RL to macroeconomic policy.