Dynamic Expert-Guided Model Averaging for Causal Discovery

TL;DR

This paper introduces a flexible, expert-guided model averaging approach for causal discovery that dynamically integrates multiple algorithms and expert input, improving performance on noisy and clean data.

Contribution

It presents a novel ensemble method that distinguishes edge existence and orientation, leveraging expert knowledge and algorithm disagreement for better causal discovery.

Findings

Outperforms strong baselines on both clean and noisy data.

Effectively leverages limited and imperfect expert input.

Utilizes disagreement among algorithms to query experts selectively.

Abstract

Would-be practitioners of causal discovery face a dizzying array of algorithms without a clear best choice. This abundance of competitive methods makes ensembling a natural strategy for practical applications. At the same time, real-world use cases frequently violate the assumptions on which common causal discovery algorithms are based, forcing reliance on expert knowledge. Inspired by recent work on dynamically requested expert knowledge and large language models (LLMs) as experts, we present a flexible model averaging method that integrates selective expert querying to ensemble a diverse set of causal discovery algorithms. Crucially, we distinguish between edge existence and orientation, enabling the method to leverage the complementary strengths of data-driven discovery and expert input. We further consider the realistic setting of limited access to an imperfect expert, using disagreement among algorithms to query the expert in cases of greater uncertainty. Experiments demonstrate that our method consistently outperforms strong baselines on both clean and noisy data. Code and data are available at https://anonymous.4open.science/r/expert-cd-ensemble-3282/.