A-Lamp: Adaptive Layout-Aware Multi-Patch Deep Convolutional Neural Network for Photo Aesthetic Assessment

TL;DR

This paper introduces A-Lamp, a neural network architecture that assesses photo aesthetics by processing images of arbitrary size, capturing both fine details and overall layout without distortion, leading to improved performance.

Contribution

The paper proposes a novel adaptive multi-patch CNN with a dual-subnet structure and feature aggregation, enabling effective aesthetic assessment of arbitrarily sized images.

Findings

Significant performance improvement over state-of-the-art methods on AVA benchmark.

Effective handling of arbitrary image sizes without distortion.

Enhanced ability to learn from both fine details and holistic layout.

Abstract

Deep convolutional neural networks (CNN) have recently been shown to generate promising results for aesthetics assessment. However, the performance of these deep CNN methods is often compromised by the constraint that the neural network only takes the fixed-size input. To accommodate this requirement, input images need to be transformed via cropping, warping, or padding, which often alter image composition, reduce image resolution, or cause image distortion. Thus the aesthetics of the original images is impaired because of potential loss of fine grained details and holistic image layout. However, such fine grained details and holistic image layout is critical for evaluating an image's aesthetics. In this paper, we present an Adaptive Layout-Aware Multi-Patch Convolutional Neural Network (A-Lamp CNN) architecture for photo aesthetic assessment. This novel scheme is able to accept arbitrary sized images, and learn from both fined grained details and holistic image layout simultaneously. To enable training on these hybrid inputs, we extend the method by developing a dedicated double-subnet neural network structure, i.e. a Multi-Patch subnet and a Layout-Aware subnet. We further construct an aggregation layer to effectively combine the hybrid features from these two subnets. Extensive experiments on the large-scale aesthetics assessment benchmark (AVA) demonstrate significant performance improvement over the state-of-the-art in photo aesthetic assessment.