AIA Forecaster: Technical Report
Rohan Alur, Bradly C. Stadie, Daniel Kang, Ryan Chen, Matt McManus, Michael Rickert, Tyler Lee, Michael Federici, Richard Zhu, Dennis Fogerty, Hayley Williamson, Nina Lozinski, Aaron Linsky, Jasjeet S. Sekhon

TL;DR
The AIA Forecaster is a large language model-based system that achieves expert-level judgmental forecasting performance, combining search, reconciliation, and calibration techniques, and outperforms prior models on benchmark datasets.
Contribution
This work introduces the AIA Forecaster, the first LLM-based system to verifiably reach expert-level forecasting performance at scale, with novel integration of search, reconciliation, and calibration.
Findings
Achieves performance equal to human superforecasters on ForecastBench.
Outperforms prior LLM baselines in forecasting accuracy.
Ensemble with market data improves overall forecasting performance.
Abstract
This technical report describes the AIA Forecaster, a Large Language Model (LLM)-based system for judgmental forecasting using unstructured data. The AIA Forecaster approach combines three core elements: agentic search over high-quality news sources, a supervisor agent that reconciles disparate forecasts for the same event, and a set of statistical calibration techniques to counter behavioral biases in large language models. On the ForecastBench benchmark (Karger et al., 2024), the AIA Forecaster achieves performance equal to human superforecasters, surpassing prior LLM baselines. In addition to reporting on ForecastBench, we also introduce a more challenging forecasting benchmark sourced from liquid prediction markets. While the AIA Forecaster underperforms market consensus on this benchmark, an ensemble combining AIA Forecaster with market consensus outperforms consensus alone,…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsForecasting Techniques and Applications · Stock Market Forecasting Methods · Explainable Artificial Intelligence (XAI)
