The Promise and Peril of Generative AI: Evidence from GPT as Sell-Side Analysts

TL;DR

This paper investigates GPT's performance as a financial analyst, revealing that while its narrative understanding is consistent, its numerical reasoning varies, and proposes a diagnostic framework to assess forecast reliability and inform investor caution.

Contribution

It introduces a diagnostic framework linking GPT's forecast accuracy to observable features, highlighting new information frictions in AI-mediated financial analysis.

Findings

GPT's narrative focus is human-like but not always accurate.

Numerical reasoning by GPT varies significantly across contexts.

Proposes indicators for assessing forecast reliability and investor caution.

Abstract

Large language models (LLMs) promise to democratize financial analysis by reducing information-processing costs. Yet equal access does not ensure equal outcomes, as the locus of friction may shift from processing information to evaluating model outputs. We study GPT's earnings forecasts following corporate earnings releases and document two patterns. First, GPT's narrative attention is consistent and human-like but not always associated with higher forecast accuracy. Second, its quantitative reasoning varies substantially across contexts, challenging the view that LLMs are uniformly weak at numerical tasks. Building on these insights, we propose a diagnostic framework that links forecast accuracy to observable processing features (i.e., narrative focus, numerical reasoning, and self-assessed confidence). These indicators serve as proxies for this new form of information friction and alert investors when to exercise caution. Our study has implications for information frictions, regulatory oversight, and the economics of AI-mediated financial markets.