Deep Relative Attributes

TL;DR

This paper presents a deep neural network approach for relative attribute prediction in images, outperforming previous methods by learning effective features end-to-end with a ranking loss.

Contribution

It introduces a convolutional neural network with a ranking layer trained end-to-end for relative attribute prediction, improving accuracy over prior handcrafted feature methods.

Findings

Outperforms baseline and state-of-the-art methods

Learns effective attribute-specific features

Provides visualizations demonstrating learned saliency maps

Abstract

Visual attributes are great means of describing images or scenes, in a way both humans and computers understand. In order to establish a correspondence between images and to be able to compare the strength of each property between images, relative attributes were introduced. However, since their introduction, hand-crafted and engineered features were used to learn increasingly complex models for the problem of relative attributes. This limits the applicability of those methods for more realistic cases. We introduce a deep neural network architecture for the task of relative attribute prediction. A convolutional neural network (ConvNet) is adopted to learn the features by including an additional layer (ranking layer) that learns to rank the images based on these features. We adopt an appropriate ranking loss to train the whole network in an end-to-end fashion. Our proposed method outperforms the baseline and state-of-the-art methods in relative attribute prediction on various coarse and fine-grained datasets. Our qualitative results along with the visualization of the saliency maps show that the network is able to learn effective features for each specific attribute. Source code of the proposed method is available at https://github.com/yassersouri/ghiaseddin.