Learning to Look before Learning to Like: Incorporating Human Visual Cognition into Aesthetic Quality Assessment

TL;DR

This paper introduces AestheticNet, a new aesthetic quality assessment model inspired by human visual cognition, which improves prediction accuracy by integrating gaze-based visual attention with semantic perception.

Contribution

The paper proposes a cognitive-inspired two-pathway architecture that incorporates human-like visual attention into aesthetic quality assessment, demonstrating improved performance and modularity.

Findings

Gaze-aligned visual attention enhances aesthetic prediction accuracy.

The gaze module acts as a model-agnostic corrector for diverse AQA backbones.

Experiments confirm the importance of human-like visual cognition in AQA.

Abstract

Automated Aesthetic Quality Assessment (AQA) treats images primarily as static pixel vectors, aligning predictions with human-rating scores largely through semantic perception. However, this paradigm diverges from human aesthetic cognition, which arises from dynamic visual exploration shaped by scanning paths, processing fluency, and the interplay between bottom-up salience and top-down intention. We introduce AestheticNet, a novel cognitive-inspired AQA paradigm that integrates human-like visual cognition and semantic perception with a two-pathway architecture. The visual attention pathway, implemented as a gaze-aligned visual encoder (GAVE) pre-trained offline on eye-tracking data using resource-efficient contrast gaze alignment, models attention from human vision system. This pathway augments the semantic pathway, which uses a fixed semantic encoder such as CLIP, through cross-attention fusion. Visual attention provides a cognitive prior reflecting foreground/background structure, color cascade, brightness, and lighting, all of which are determinants of aesthetic perception beyond semantics. Experiments validated by hypothesis testing show a consistent improvement over the semantic-alone baselines, and demonstrate the gaze module as a model-agnostic corrector compatible with diverse AQA backbones, supporting the necessity and modularity of human-like visual cognition for AQA. Our code is available at https://github.com/keepgallop/AestheticNet.