iTAG: Inverse Design for Natural Text Generation with Accurate Causal Graph Annotations

TL;DR

iTAG is a novel method that generates natural text with accurate causal graph annotations by iteratively refining concept assignments, enabling scalable benchmarking of causal discovery algorithms.

Contribution

The paper introduces iTAG, a new inverse problem approach that improves causal annotation accuracy in text generation using LLMs, balancing naturalness and correctness.

Findings

iTAG achieves high causal annotation accuracy and naturalness.

Generated data correlates well with real-world causal data.

iTAG enables scalable benchmarking of causal discovery algorithms.

Abstract

A fundamental obstacle to causal discovery from text is the lack of causally annotated text data for use as ground truth, due to high annotation costs. This motivates an important task of generating text with causal graph annotations. Early template-based generation methods sacrifice text naturalness in exchange for high causal graph annotation accuracy. Recent Large Language Model (LLM)-dependent methods directly generate natural text from target graphs through LLMs, but do not guarantee causal graph annotation accuracy. Therefore, we propose iTAG, which performs real-world concept assignment to nodes before converting causal graphs into text in existing LLM-dependent methods. iTAG frames this process as an inverse problem with the causal graph as the target, iteratively examining and refining concept selection through Chain-of-Thought (CoT) reasoning so that the induced relations between concepts are as consistent as possible with the target causal relationships described by the causal graph. iTAG demonstrates both extremely high annotation accuracy and naturalness across extensive tests, and the results of testing text-based causal discovery algorithms with the generated data show high statistical correlation with real-world data. This suggests that iTAG-generated data can serve as a practical surrogate for scalable benchmarking of text-based causal discovery algorithms.