Reinforcement Learning and Consumption-Savings Behavior

TL;DR

This paper uses reinforcement learning models with neural networks to explain household consumption patterns during downturns, capturing empirical phenomena like high MPCs among unemployed and persistent consumption scarring.

Contribution

It introduces a reinforcement learning framework with neural network approximation to explain consumption behaviors, departing from traditional rational expectations models.

Findings

Reproduces higher MPCs for low-asset unemployed households.

Captures persistent consumption scarring after unemployment.

Aligns simulation results with empirical estimates.

Abstract

This paper demonstrates how reinforcement learning can explain two puzzling empirical patterns in household consumption behavior during economic downturns. I develop a model where agents use Q-learning with neural network approximation to make consumption-savings decisions under income uncertainty, departing from standard rational expectations assumptions. The model replicates two key findings from recent literature: (1) unemployed households with previously low liquid assets exhibit substantially higher marginal propensities to consume (MPCs) out of stimulus transfers compared to high-asset households (0.50 vs 0.34), even when neither group faces borrowing constraints, consistent with Ganong et al. (2024); and (2) households with more past unemployment experiences maintain persistently lower consumption levels after controlling for current economic conditions, a "scarring" effect documented by Malmendier and Shen (2024). Unlike existing explanations based on belief updating about income risk or ex-ante heterogeneity, the reinforcement learning mechanism generates both higher MPCs and lower consumption levels simultaneously through value function approximation errors that evolve with experience. Simulation results closely match the empirical estimates, suggesting that adaptive learning through reinforcement learning provides a unifying framework for understanding how past experiences shape current consumption behavior beyond what current economic conditions would predict.