LLM Generated Distribution-Based Prediction of US Electoral Results, Part I

TL;DR

This paper proposes a novel distribution-based prediction method using Large Language Models to analyze and improve the reliability and transparency of electoral outcome predictions by interpreting token probabilities as distributions.

Contribution

It introduces distribution-based prediction as a new approach for leveraging LLMs in electoral forecasting, providing insights into bias, noise, and fidelity.

Findings

Effective in predicting US presidential election results

Identifies task-specific biases and prompt noise

Enhances understanding of LLM reliability

Abstract

This paper introduces distribution-based prediction, a novel approach to using Large Language Models (LLMs) as predictive tools by interpreting output token probabilities as distributions representing the models' learned representation of the world. This distribution-based nature offers an alternative perspective for analyzing algorithmic fidelity, complementing the approach used in silicon sampling. We demonstrate the use of distribution-based prediction in the context of recent United States presidential election, showing that this method can be used to determine task specific bias, prompt noise, and algorithmic fidelity. This approach has significant implications for assessing the reliability and increasing transparency of LLM-based predictions across various domains.