Learning from zero: how to make consumption-saving decisions in a stochastic environment with an AI algorithm

TL;DR

This paper introduces an AI-based learning mechanism for economic agents to make adaptive consumption-saving decisions in stochastic environments, combining reinforcement learning with economic modeling.

Contribution

It develops a novel actor-critic AI framework for economic decision-making, integrating exploration and learning in stochastic macroeconomic models.

Findings

AI agents adapt to economic shocks through exploration.

Different exploration levels lead to varied welfare outcomes.

The model generalizes to broader economic decision-making contexts.

Abstract

This exercise proposes a learning mechanism to model economic agent's decision-making process using an actor-critic structure in the literature of artificial intelligence. It is motivated by the psychology literature of learning through reinforcing good or bad decisions. In a model of an environment, to learn to make decisions, this AI agent needs to interact with its environment and make explorative actions. Each action in a given state brings a reward signal to the agent. These interactive experience is saved in the agent's memory, which is then used to update its subjective belief of the world. The agent's decision-making strategy is formed and adjusted based on this evolving subjective belief. This agent does not only take an action that it knows would bring a high reward, it also explores other possibilities. This is the process of taking explorative actions, and it ensures that the agent notices changes in its environment and adapt its subjective belief and decisions accordingly. Through a model of stochastic optimal growth, I illustrate that the economic agent under this proposed learning structure is adaptive to changes in an underlying stochastic process of the economy. AI agents can differ in their levels of exploration, which leads to different experience in the same environment. This reflects on to their different learning behaviours and welfare obtained. The chosen economic structure possesses the fundamental decision making problems of macroeconomic models, i.e., how to make consumption-saving decisions in a lifetime, and it can be generalised to other decision-making processes and economic models.