Reasoning in Computer Vision: Taxonomy, Models, Tasks, and Methodologies

TL;DR

This survey categorizes and analyzes various types of visual reasoning in computer vision, examining models, tasks, evaluation protocols, and open challenges to guide future research towards more integrated and trustworthy AI systems.

Contribution

It provides a comprehensive taxonomy of visual reasoning types, reviews diverse models and evaluation methods, and identifies key open challenges for advancing the field.

Findings

Relational, symbolic, temporal, causal, and commonsense reasoning are systematically categorized.

Evaluation protocols are critically analyzed for their limitations in generalizability and reproducibility.

Open challenges include scalability, integration of paradigms, and lack of comprehensive benchmarks.

Abstract

Visual reasoning is critical for a wide range of computer vision tasks that go beyond surface-level object detection and classification. Despite notable advances in relational, symbolic, temporal, causal, and commonsense reasoning, existing surveys often address these directions in isolation, lacking a unified analysis and comparison across reasoning types, methodologies, and evaluation protocols. This survey aims to address this gap by categorizing visual reasoning into five major types (relational, symbolic, temporal, causal, and commonsense) and systematically examining their implementation through architectures such as graph-based models, memory networks, attention mechanisms, and neuro-symbolic systems. We review evaluation protocols designed to assess functional correctness, structural consistency, and causal validity, and critically analyze their limitations in terms of generalizability, reproducibility, and explanatory power. Beyond evaluation, we identify key open challenges in visual reasoning, including scalability to complex scenes, deeper integration of symbolic and neural paradigms, the lack of comprehensive benchmark datasets, and reasoning under weak supervision. Finally, we outline a forward-looking research agenda for next-generation vision systems, emphasizing that bridging perception and reasoning is essential for building transparent, trustworthy, and cross-domain adaptive AI systems, particularly in critical domains such as autonomous driving and medical diagnostics.