On the Role of Individual Differences in Current Approaches to Computational Image Aesthetics

TL;DR

This paper develops a theoretical framework for image aesthetic assessment, highlighting how transfer learning between generic and personal models involves extrapolation and interpolation, and shows that individual differences significantly impact model performance.

Contribution

It introduces a unified model encoding individual traits and provides a theoretical analysis of transfer learning effects in IAA, supported by extensive experiments.

Findings

Transfer from GIAA to PIAA involves extrapolation, reverse involves interpolation.

Significant performance variation exists even in GIAA, challenging averaging assumptions.

Education, photography, and art experience influence aesthetic judgments.

Abstract

Image aesthetic assessment (IAA) evaluates image aesthetics, a task complicated by image diversity and user subjectivity. Current approaches address this in two stages: Generic IAA (GIAA) models estimate mean aesthetic scores, while Personal IAA (PIAA) models adapt GIAA using transfer learning to incorporate user subjectivity. However, a theoretical understanding of transfer learning between GIAA and PIAA, particularly concerning the impact of group composition, group size, aesthetic differences between groups and individuals, and demographic correlations, is lacking. This work establishes a theoretical foundation for IAA, proposing a unified model that encodes individual characteristics in a distributional format for both individual and group assessments. We show that transferring from GIAA to PIAA involves extrapolation, while the reverse involves interpolation, which is generally more effective for machine learning. Extensive experiments with varying group compositions, including sub-sampling by group size and disjoint demographics, reveal substantial performance variation even for GIAA, challenging the assumption that averaging scores eliminates individual subjectivity. Score-distribution analysis using Earth Mover's Distance (EMD) and the Gini index identifies education, photography experience, and art experience as key factors in aesthetic differences, with greater subjectivity in artworks than in photographs. Code is available at https://github.com/lwchen6309/aesthetics_transfer_learning.