TCD-Arena: Assessing Robustness of Time Series Causal Discovery Methods Against Assumption Violations

TL;DR

TCD-Arena is a comprehensive testing toolkit designed to evaluate the robustness of time series causal discovery methods under various assumption violations, aiding in the development of more reliable algorithms.

Contribution

It introduces a modular, customizable testing framework and provides extensive empirical analysis of 33 assumption violations affecting causal discovery methods.

Findings

Encompasses around 30 million CD attempts revealing robustness profiles.

Identifies that ensemble methods can enhance robustness against assumption violations.

Provides insights to improve the reliability of causal discovery in diverse data conditions.

Abstract

Causal Discovery (CD) is a powerful framework for scientific inquiry. Yet, its practical adoption is hindered by a reliance on strong, often unverifiable assumptions and a lack of robust performance assessment. To address these limitations and advance empirical CD evaluation, we present TCD-Arena, a modularized, highly customizable, and extendable testing kit to assess the robustness of time series CD algorithms against stepwise more severe assumption violations. For demonstration, we conduct an extensive empirical study comprising around 30 million individual CD attempts and reveal nuanced robustness profiles for 33 distinct assumption violations. Further, we investigate CD ensembles and find that they have the potential to improve general robustness, which has implications for real-world applications. With this, we strive to ultimately facilitate the development of CD methods that are reliable for a diverse range of synthetic and potentially real-world data conditions.