Automatic Image Cropping for Visual Aesthetic Enhancement Using Deep Neural Networks and Cascaded Regression

TL;DR

This paper introduces a cascaded regression approach using deep neural networks to automate image cropping, aiming to enhance visual aesthetics by learning from professional photographers and large-scale aesthetic datasets.

Contribution

It presents a novel CCR method with improved convergence and a two-step learning strategy to effectively predict aesthetically pleasing cropping bounding boxes.

Findings

Outperforms state-of-the-art image cropping methods

Demonstrates significant improvement in aesthetic quality of cropped images

Efficiently learns from large-scale aesthetic datasets

Abstract

Despite recent progress, computational visual aesthetic is still challenging. Image cropping, which refers to the removal of unwanted scene areas, is an important step to improve the aesthetic quality of an image. However, it is challenging to evaluate whether cropping leads to aesthetically pleasing results because the assessment is typically subjective. In this paper, we propose a novel cascaded cropping regression (CCR) method to perform image cropping by learning the knowledge from professional photographers. The proposed CCR method improves the convergence speed of the cascaded method, which directly uses random-ferns regressors. In addition, a two-step learning strategy is proposed and used in the CCR method to address the problem of lacking labelled cropping data. Specifically, a deep convolutional neural network (CNN) classifier is first trained on large-scale visual aesthetic datasets. The deep CNN model is then designed to extract features from several image cropping datasets, upon which the cropping bounding boxes are predicted by the proposed CCR method. Experimental results on public image cropping datasets demonstrate that the proposed method significantly outperforms several state-of-the-art image cropping methods.