GPT has become financially literate: Insights from financial literacy tests of GPT and a preliminary test of how people use it as a source of advice

TL;DR

This paper evaluates GPT's financial literacy through tests, revealing that advanced models like GPT-4 demonstrate near-human or superhuman financial understanding, and explores how people might use GPT as a financial advice source.

Contribution

It provides empirical evidence that state-of-the-art GPT models exhibit emergent financial literacy and introduces methods to assess advice utilization from large language models.

Findings

GPT-3.5 scores 66-65% on financial literacy tests.

GPT-4 achieves 99% on the same test.

Potential for GPT to serve as a financial advice source.

Abstract

We assess the ability of GPT -- a large language model -- to serve as a financial robo-advisor for the masses, by using a financial literacy test. Davinci and ChatGPT based on GPT-3.5 score 66% and 65% on the financial literacy test, respectively, compared to a baseline of 33%. However, ChatGPT based on GPT-4 achieves a near-perfect 99% score, pointing to financial literacy becoming an emergent ability of state-of-the-art models. We use the Judge-Advisor System and a savings dilemma to illustrate how researchers might assess advice-utilization from large language models. We also present a number of directions for future research.