Effective Aesthetics Prediction with Multi-level Spatially Pooled Features

TL;DR

This paper introduces a novel deep learning method for aesthetics prediction that utilizes full-resolution images and multi-level spatially pooled features, significantly outperforming previous models on the AVA dataset.

Contribution

It presents the first approach supporting full-resolution images with variable input sizes, improving aesthetics assessment accuracy using MLSP features from a pre-trained network.

Findings

Achieved a SRCC of 0.756, surpassing the previous best of 0.612.

Supports full-resolution images, capturing more image information.

Uses MLSP features from all convolutional blocks of InceptionResNet-v2.

Abstract

We propose an effective deep learning approach to aesthetics quality assessment that relies on a new type of pre-trained features, and apply it to the AVA data set, the currently largest aesthetics database. While previous approaches miss some of the information in the original images, due to taking small crops, down-scaling or warping the originals during training, we propose the first method that efficiently supports full resolution images as an input, and can be trained on variable input sizes. This allows us to significantly improve upon the state of the art, increasing the Spearman rank-order correlation coefficient (SRCC) of ground-truth mean opinion scores (MOS) from the existing best reported of 0.612 to 0.756. To achieve this performance, we extract multi-level spatially pooled (MLSP) features from all convolutional blocks of a pre-trained InceptionResNet-v2 network, and train a custom shallow Convolutional Neural Network (CNN) architecture on these new features.