CausalVLR: A Toolbox and Benchmark for Visual-Linguistic Causal Reasoning

TL;DR

CausalVLR is an open-source toolbox that offers a comprehensive set of causal reasoning methods and benchmarks for visual-linguistic tasks, facilitating research and development in causal inference.

Contribution

It introduces the most complete visual-linguistic causal reasoning toolbox with PyTorch implementations, model weights, and a benchmark for various tasks.

Findings

Includes state-of-the-art causal relation discovery methods

Supports multiple visual-linguistic tasks like VQA and captioning

Provides a flexible platform for developing new causal reasoning methods

Abstract

We present CausalVLR (Causal Visual-Linguistic Reasoning), an open-source toolbox containing a rich set of state-of-the-art causal relation discovery and causal inference methods for various visual-linguistic reasoning tasks, such as VQA, image/video captioning, medical report generation, model generalization and robustness, etc. These methods have been included in the toolbox with PyTorch implementations under NVIDIA computing system. It not only includes training and inference codes, but also provides model weights. We believe this toolbox is by far the most complete visual-linguitic causal reasoning toolbox. We wish that the toolbox and benchmark could serve the growing research community by providing a flexible toolkit to re-implement existing methods and develop their own new causal reasoning methods. Code and models are available at https://github.com/HCPLab-SYSU/CausalVLR. The project is under active development by HCP-Lab's contributors and we will keep this document updated.