Scalability of Bayesian Network Structure Elicitation with Large Language Models: a Novel Methodology and Comparative Analysis

TL;DR

This paper introduces a new LLM-based approach for Bayesian Network structure elicitation, compares it with existing methods, and examines the scalability and limitations of using LLMs for this task across various network sizes.

Contribution

The paper presents a novel LLM-based methodology for BN structure elicitation, including a voting mechanism and contamination check, and provides a comparative analysis of scalability and applicability.

Findings

Our method outperforms existing methods with one LLM on certain BNs.

Performance declines as BN size increases.

Some BNs are unsuitable for LLM-based structure elicitation due to node naming issues.

Abstract

In this work, we propose a novel method for Bayesian Networks (BNs) structure elicitation that is based on the initialization of several LLMs with different experiences, independently querying them to create a structure of the BN, and further obtaining the final structure by majority voting. We compare the method with one alternative method on various widely and not widely known BNs of different sizes and study the scalability of both methods on them. We also propose an approach to check the contamination of BNs in LLM, which shows that some widely known BNs are inapplicable for testing the LLM usage for BNs structure elicitation. We also show that some BNs may be inapplicable for such experiments because their node names are indistinguishable. The experiments on the other BNs show that our method performs better than the existing method with one of the three studied LLMs; however, the performance of both methods significantly decreases with the increase in BN size.