Clothing Retrieval with Visual Attention Model

TL;DR

This paper introduces a self-learning Visual Attention Model for clothing retrieval that enhances accuracy and robustness by adaptively focusing on clothing regions without requiring strong supervision.

Contribution

It proposes a novel end-to-end attention mechanism with Impdrop connection, reducing supervision needs and improving performance on clothing retrieval benchmarks.

Findings

The VAM improves retrieval accuracy on benchmark datasets.

Impdrop connection enhances robustness with limited training data.

The method outperforms existing landmark-based approaches.

Abstract

Clothing retrieval is a challenging problem in computer vision. With the advance of Convolutional Neural Networks (CNNs), the accuracy of clothing retrieval has been significantly improved. FashionNet[1], a recent study, proposes to employ a set of artificial features in the form of landmarks for clothing retrieval, which are shown to be helpful for retrieval. However, the landmark detection module is trained with strong supervision which requires considerable efforts to obtain. In this paper, we propose a self-learning Visual Attention Model (VAM) to extract attention maps from clothing images. The VAM is further connected to a global network to form an end-to-end network structure through Impdrop connection which randomly Dropout on the feature maps with the probabilities given by the attention map. Extensive experiments on several widely used benchmark clothing retrieval data sets have demonstrated the promise of the proposed method. We also show that compared to the trivial Product connection, the Impdrop connection makes the network structure more robust when training sets of limited size are used.