A deep architecture for unified aesthetic prediction

TL;DR

This paper introduces a deep learning model that predicts aesthetic score distributions of images, outperforming previous methods on multiple tasks and providing insights into aesthetic modifications.

Contribution

The work presents a novel architecture and training method for predicting aesthetic score distributions, a less-studied aspect in image aesthetics prediction.

Findings

Achieves state-of-the-art results on AVA dataset for three aesthetic tasks.

Uses a single model trained for distribution prediction to perform classification and regression.

Introduces a method to modify images based on aesthetic predictions to understand model behavior.

Abstract

Image aesthetics has become an important criterion for visual content curation on social media sites and media content repositories. Previous work on aesthetic prediction models in the computer vision community has focused on aesthetic score prediction or binary image labeling. However, raw aesthetic annotations are in the form of score histograms and provide richer and more precise information than binary labels or mean scores. Consequently, in this work we focus on the rarely-studied problem of predicting aesthetic score distributions and propose a novel architecture and training procedure for our model. Our model achieves state-of-the-art results on the standard AVA large-scale benchmark dataset for three tasks: (i) aesthetic quality classification; (ii) aesthetic score regression; and (iii) aesthetic score distribution prediction, all while using one model trained only for the distribution prediction task. We also introduce a method to modify an image such that its predicted aesthetics changes, and use this modification to gain insight into our model.