An Universal Image Attractiveness Ranking Framework

TL;DR

This paper introduces a deep learning framework for ranking image attractiveness based on pairwise comparisons, improving search result quality with fewer judgments and adapting to individual preferences.

Contribution

A novel pairwise deep network model for image attractiveness ranking that learns from relative judgments and personal preferences, outperforming existing models.

Findings

Outperforms state-of-the-art models on web test data.

Requires significantly fewer judgments than previous methods.

Enhances search engine results using attractiveness information.

Abstract

We propose a new framework to rank image attractiveness using a novel pairwise deep network trained with a large set of side-by-side multi-labeled image pairs from a web image index. The judges only provide relative ranking between two images without the need to directly assign an absolute score, or rate any predefined image attribute, thus making the rating more intuitive and accurate. We investigate a deep attractiveness rank net (DARN), a combination of deep convolutional neural network and rank net, to directly learn an attractiveness score mean and variance for each image and the underlying criteria the judges use to label each pair. The extension of this model (DARN-V2) is able to adapt to individual judge's personal preference. We also show the attractiveness of search results are significantly improved by using this attractiveness information in a real commercial search engine. We evaluate our model against other state-of-the-art models on our side-by-side web test data and another public aesthetic data set. With much less judgments (1M vs 50M), our model outperforms on side-by-side labeled data, and is comparable on data labeled by absolute score.