Aesthetic Attributes Assessment of Images with AMANv2 and DPC-CaptionsV2

TL;DR

This paper introduces a new dataset and a multi-attribute network for assessing and captioning aesthetic attributes of images, improving the accuracy of aesthetic comment prediction.

Contribution

It presents a novel dataset, DPC-CaptionsV2, and an improved AMANv2 model for aesthetic attribute captioning, advancing image aesthetic assessment methods.

Findings

AMANv2 outperforms previous models in predicting aesthetic attributes.

The dataset enables better training of aesthetic captioning models.

The method produces comments closer to human aesthetic perceptions.

Abstract

Image aesthetic quality assessment is popular during the last decade. Besides numerical assessment, nature language assessment (aesthetic captioning) has been proposed to describe the generally aesthetic impression of an image. In this paper, we propose aesthetic attribute assessment, which is the aesthetic attributes captioning, i.e., to assess the aesthetic attributes such as composition, lighting usage and color arrangement. It is a non-trivial task to label the comments of aesthetic attributes, which limit the scale of the corresponding datasets. We construct a novel dataset, named DPC-CaptionsV2, by a semi-automatic way. The knowledge is transferred from a small-scale dataset with full annotations to large-scale professional comments from a photography website. Images of DPC-CaptionsV2 contain comments up to 4 aesthetic attributes: composition, lighting, color, and subject. Then, we propose a new version of Aesthetic Multi-Attributes Networks (AMANv2) based on the BUTD model and the VLPSA model. AMANv2 fuses features of a mixture of small-scale PCCD dataset with full annotations and large-scale DPCCaptionsV2 dataset with full annotations. The experimental results of DPCCaptionsV2 show that our method can predict the comments on 4 aesthetic attributes, which are closer to aesthetic topics than those produced by the previous AMAN model. Through the evaluation criteria of image captioning, the specially designed AMANv2 model is better to the CNN-LSTM model and the AMAN model.