An Explainable AI Approach to Large Language Model Assisted Causal Model Auditing and Development

TL;DR

This paper introduces a novel approach using large language models like ChatGPT to audit and improve causal networks, aiding human analysts in identifying errors and refining models.

Contribution

It proposes leveraging LLMs as an interactive auditing tool for causal networks, integrating automated inference with human expertise for better model accuracy.

Findings

ChatGPT can suggest edge directionality and confounders.

Visual summaries assist human decision-making.

Prototype shows promising initial results.

Abstract

Causal networks are widely used in many fields, including epidemiology, social science, medicine, and engineering, to model the complex relationships between variables. While it can be convenient to algorithmically infer these models directly from observational data, the resulting networks are often plagued with erroneous edges. Auditing and correcting these networks may require domain expertise frequently unavailable to the analyst. We propose the use of large language models such as ChatGPT as an auditor for causal networks. Our method presents ChatGPT with a causal network, one edge at a time, to produce insights about edge directionality, possible confounders, and mediating variables. We ask ChatGPT to reflect on various aspects of each causal link and we then produce visualizations that summarize these viewpoints for the human analyst to direct the edge, gather more data, or test further hypotheses. We envision a system where large language models, automated causal inference, and the human analyst and domain expert work hand in hand as a team to derive holistic and comprehensive causal models for any given case scenario. This paper presents first results obtained with an emerging prototype.