Exploring Multi-Modal Data with Tool-Augmented LLM Agents for Precise Causal Discovery

TL;DR

This paper introduces MATMCD, a multi-agent system using tool-augmented LLMs to enhance causal discovery by leveraging multi-modal data, showing promising results across diverse datasets.

Contribution

It presents a novel multi-agent framework that combines data augmentation and causal reasoning with multi-modal data for improved causal discovery.

Findings

Significant potential demonstrated across seven datasets.

Multi-modal data improves causal discovery accuracy.

Effective cooperation of agents enhances reasoning capabilities.

Abstract

Causal discovery is an imperative foundation for decision-making across domains, such as smart health, AI for drug discovery and AIOps. Traditional statistical causal discovery methods, while well-established, predominantly rely on observational data and often overlook the semantic cues inherent in cause-and-effect relationships. The advent of Large Language Models (LLMs) has ushered in an affordable way of leveraging the semantic cues for knowledge-driven causal discovery, but the development of LLMs for causal discovery lags behind other areas, particularly in the exploration of multi-modal data. To bridge the gap, we introduce MATMCD, a multi-agent system powered by tool-augmented LLMs. MATMCD has two key agents: a Data Augmentation agent that retrieves and processes modality-augmented data, and a Causal Constraint agent that integrates multi-modal data for knowledge-driven reasoning. The proposed design of the inner-workings ensures successful cooperation of the agents. Our empirical study across seven datasets suggests the significant potential of multi-modality enhanced causal discovery.