Introducing CausalBench: A Flexible Benchmark Framework for Causal Analysis and Machine Learning

TL;DR

CausalBench is a comprehensive, open platform designed to standardize and facilitate the evaluation of causal learning algorithms, datasets, and metrics, addressing key challenges in advancing causal inference research from observational data.

Contribution

The paper introduces CausalBench, a unified benchmark framework that supports causal learning research with datasets, algorithms, metrics, and evaluation tools, promoting transparency and reproducibility.

Findings

Provides a standardized platform for causal learning evaluation

Enables fair comparison of algorithms and metrics

Facilitates collaboration and reproducibility in causal research

Abstract

While witnessing the exceptional success of machine learning (ML) technologies in many applications, users are starting to notice a critical shortcoming of ML: correlation is a poor substitute for causation. The conventional way to discover causal relationships is to use randomized controlled experiments (RCT); in many situations, however, these are impractical or sometimes unethical. Causal learning from observational data offers a promising alternative. While being relatively recent, causal learning aims to go far beyond conventional machine learning, yet several major challenges remain. Unfortunately, advances are hampered due to the lack of unified benchmark datasets, algorithms, metrics, and evaluation service interfaces for causal learning. In this paper, we introduce {\em CausalBench}, a transparent, fair, and easy-to-use evaluation platform, aiming to (a) enable the advancement of research in causal learning by facilitating scientific collaboration in novel algorithms, datasets, and metrics and (b) promote scientific objectivity, reproducibility, fairness, and awareness of bias in causal learning research. CausalBench provides services for benchmarking data, algorithms, models, and metrics, impacting the needs of a broad of scientific and engineering disciplines.