Visual Categorization Across Minds and Models: Cognitive Analysis of Human Labeling and Neuro-Symbolic Integration

TL;DR

This paper compares human and AI visual labeling strategies using cognitive science frameworks, revealing parallels and differences, and proposes neuro-symbolic models for more interpretable and cognitively aligned AI systems.

Contribution

It introduces a comprehensive cognitive analysis of human and AI visual interpretation, integrating neuro-symbolic models grounded in cognitive principles.

Findings

Humans use heuristic, layered decision strategies under uncertainty.

AI models focus on feature-based processing with attention visualization.

Neuro-symbolic architectures can unify symbolic reasoning with connectionist representations.

Abstract

Understanding how humans and AI systems interpret ambiguous visual stimuli offers critical insight into the nature of perception, reasoning, and decision-making. This paper examines image labeling performance across human participants and deep neural networks, focusing on low-resolution, perceptually degraded stimuli. Drawing from computational cognitive science, cognitive architectures, and connectionist-symbolic hybrid models, we contrast human strategies such as analogical reasoning, shape-based recognition, and confidence modulation with AI's feature-based processing. Grounded in Marr's tri-level hypothesis, Simon's bounded rationality, and Thagard's frameworks of representation and emotion, we analyze participant responses in relation to Grad-CAM visualizations of model attention. Human behavior is further interpreted through cognitive principles modeled in ACT-R and Soar, revealing layered and heuristic decision strategies under uncertainty. Our findings highlight key parallels and divergences between biological and artificial systems in representation, inference, and confidence calibration. The analysis motivates future neuro-symbolic architectures that unify structured symbolic reasoning with connectionist representations. Such architectures, informed by principles of embodiment, explainability, and cognitive alignment, offer a path toward AI systems that are not only performant but also interpretable and cognitively grounded.