LLM-BI: Towards Fully Automated Bayesian Inference with Large Language Models

TL;DR

This paper explores using large language models to automate Bayesian inference by eliciting priors and specifying models from natural language, aiming to simplify and democratize Bayesian workflows.

Contribution

It introduces LLM-BI, a pipeline that leverages LLMs to automate prior elicitation and model specification in Bayesian inference, demonstrated through linear regression experiments.

Findings

LLMs can successfully extract priors from natural language.

LLMs can specify complete Bayesian models from high-level descriptions.

Potential for fully automated Bayesian inference pipelines.

Abstract

A significant barrier to the widespread adoption of Bayesian inference is the specification of prior distributions and likelihoods, which often requires specialized statistical expertise. This paper investigates the feasibility of using a Large Language Model (LLM) to automate this process. We introduce LLM-BI (Large Language Model-driven Bayesian Inference), a conceptual pipeline for automating Bayesian workflows. As a proof-of-concept, we present two experiments focused on Bayesian linear regression. In Experiment I, we demonstrate that an LLM can successfully elicit prior distributions from natural language. In Experiment II, we show that an LLM can specify the entire model structure, including both priors and the likelihood, from a single high-level problem description. Our results validate the potential of LLMs to automate key steps in Bayesian modeling, enabling the possibility of an automated inference pipeline for probabilistic programming.