Complexity in Complexity: Understanding Visual Complexity Through Structure, Color, and Surprise

TL;DR

This paper investigates human perception of visual complexity by proposing interpretable features capturing structure, color, and surprise, revealing that complexity assessment requires multiple perceptual and semantic factors beyond existing models.

Contribution

It introduces new interpretable features for visual complexity, demonstrating their effectiveness and highlighting the importance of perceptual and semantic factors in complexity modeling.

Findings

Proposed Multi-Scale Sobel Gradient and Unique Color features improve complexity prediction.

Surprise scores from a Large Language Model enhance understanding of visual surprise.

Model performance indicates complexity perception involves multiple perceptual and semantic factors.

Abstract

Understanding how humans perceive visual complexity is a key area of study in visual cognition. Previous approaches to modeling visual complexity assessments have often resulted in intricate, difficult-to-interpret algorithms that employ numerous features or sophisticated deep learning architectures. While these complex models achieve high performance on specific datasets, they often sacrifice interpretability, making it challenging to understand the factors driving human perception of complexity. Recently (Shen, et al. 2024) proposed an interpretable segmentation-based model that accurately predicted complexity across various datasets, supporting the idea that complexity can be explained simply. In this work, we investigate the failure of their model to capture structural, color and surprisal contributions to complexity. To this end, we propose Multi-Scale Sobel Gradient (MSG) which measures spatial intensity variations, Multi-Scale Unique Color (MUC) which quantifies colorfulness across multiple scales, and surprise scores generated using a Large Language Model. We test our features on existing benchmarks and a novel dataset (Surprising Visual Genome) containing surprising images from Visual Genome. Our experiments demonstrate that modeling complexity accurately is not as simple as previously thought, requiring additional perceptual and semantic factors to address dataset biases. Our model improves predictive performance while maintaining interpretability, offering deeper insights into how visual complexity is perceived and assessed. Our code, analysis and data are available at https://github.com/Complexity-Project/Complexity-in-Complexity.