Causal reasoning in typical computer vision tasks

TL;DR

This paper reviews how causal reasoning can improve computer vision tasks by addressing the limitations of correlation-based deep learning, highlighting recent methods and future directions.

Contribution

It provides a comprehensive review of causal methods in vision and vision-language tasks, summarizing their advantages and proposing future research roadmaps.

Findings

Causal methods help avoid spurious correlations in vision tasks.

Causal paradigms enhance robustness and interpretability of models.

Future work includes extending causality to complex scenes and systems.

Abstract

Deep learning has revolutionized the field of artificial intelligence. Based on the statistical correlations uncovered by deep learning-based methods, computer vision has contributed to tremendous growth in areas like autonomous driving and robotics. Despite being the basis of deep learning, such correlation is not stable and is susceptible to uncontrolled factors. In the absence of the guidance of prior knowledge, statistical correlations can easily turn into spurious correlations and cause confounders. As a result, researchers are now trying to enhance deep learning methods with causal theory. Causal theory models the intrinsic causal structure unaffected by data bias and is effective in avoiding spurious correlations. This paper aims to comprehensively review the existing causal methods in typical vision and vision-language tasks such as semantic segmentation, object detection, and image captioning. The advantages of causality and the approaches for building causal paradigms will be summarized. Future roadmaps are also proposed, including facilitating the development of causal theory and its application in other complex scenes and systems.