Recovering Event Probabilities from Large Language Model Embeddings via Axiomatic Constraints

TL;DR

This paper introduces a method to recover coherent event probabilities from large language model embeddings by enforcing probability axioms in a latent space, improving the coherence and accuracy of the estimated probabilities.

Contribution

The authors propose a novel approach using a variational autoencoder with axiomatic constraints to derive more coherent and accurate event probabilities from LLM embeddings.

Findings

Recovered probabilities are more coherent than direct model outputs.

Estimated probabilities closely match true probabilities in experiments.

Method improves the reliability of uncertainty estimates in LLMs.

Abstract

Rational decision-making under uncertainty requires coherent degrees of belief in events. However, event probabilities generated by Large Language Models (LLMs) have been shown to exhibit incoherence, violating the axioms of probability theory. This raises the question of whether coherent event probabilities can be recovered from the embeddings used by the models. If so, those derived probabilities could be used as more accurate estimates in events involving uncertainty. To explore this question, we propose enforcing axiomatic constraints, such as the additive rule of probability theory, in the latent space learned by an extended variational autoencoder (VAE) applied to LLM embeddings. This approach enables event probabilities to naturally emerge in the latent space as the VAE learns to both reconstruct the original embeddings and predict the embeddings of semantically related events. We evaluate our method on complementary events (i.e., event A and its complement, event not-A), where the true probabilities of the two events must sum to 1. Experiment results on open-weight language models demonstrate that probabilities recovered from embeddings exhibit greater coherence than those directly reported by the corresponding models and align closely with the true probabilities.