Solomonoff-Inspired Hypothesis Ranking with LLMs for Prediction Under Uncertainty

TL;DR

This paper introduces a Solomonoff-inspired hypothesis ranking method using LLMs that improves uncertainty estimation and interpretability in AI reasoning tasks with sparse data, outperforming Bayesian Model Averaging.

Contribution

It presents a novel approach combining Solomonoff induction principles with LLMs to better evaluate multiple hypotheses under uncertainty, emphasizing simplicity and predictive fit.

Findings

Produces conservative, uncertainty-aware predictions

Spreads probability more evenly across hypotheses

Outperforms Bayesian Model Averaging in benchmark tasks

Abstract

Reasoning under uncertainty is a key challenge in AI, especially for real-world tasks, where problems with sparse data demands systematic generalisation. Existing approaches struggle to balance accuracy and simplicity when evaluating multiple candidate solutions. We propose a Solomonoff-inspired method that weights LLM-generated hypotheses by simplicity and predictive fit. Applied to benchmark (Mini-ARC) tasks, our method produces Solomonoff-weighted mixtures for per-cell predictions, yielding conservative, uncertainty-aware outputs even when hypotheses are noisy or partially incorrect. Compared to Bayesian Model Averaging (BMA), Solomonoff scoring spreads probability more evenly across competing hypotheses, while BMA concentrates weight on the most likely but potentially flawed candidates. Across tasks, this highlights the value of algorithmic information-theoretic priors for interpretable, reliable multi-hypothesis reasoning under uncertainty.