Cross-domain Image Retrieval with a Dual Attribute-aware Ranking Network

TL;DR

This paper introduces a Dual Attribute-aware Ranking Network (DARN) for cross-domain clothing image retrieval, effectively bridging the gap between online shopping images and user photos by leveraging semantic attributes and a large-scale dataset.

Contribution

The paper proposes a novel dual-network architecture with attribute-guided learning and ranking constraints, along with a large dataset for training and evaluation.

Findings

Top-20 retrieval accuracy doubled with DARN (0.570 vs. 0.268)

Attribute-guided learning improves retrieval performance

Large-scale dataset enables effective network training

Abstract

We address the problem of cross-domain image retrieval, considering the following practical application: given a user photo depicting a clothing image, our goal is to retrieve the same or attribute-similar clothing items from online shopping stores. This is a challenging problem due to the large discrepancy between online shopping images, usually taken in ideal lighting/pose/background conditions, and user photos captured in uncontrolled conditions. To address this problem, we propose a Dual Attribute-aware Ranking Network (DARN) for retrieval feature learning. More specifically, DARN consists of two sub-networks, one for each domain, whose retrieval feature representations are driven by semantic attribute learning. We show that this attribute-guided learning is a key factor for retrieval accuracy improvement. In addition, to further align with the nature of the retrieval problem, we impose a triplet visual similarity constraint for learning to rank across the two sub-networks. Another contribution of our work is a large-scale dataset which makes the network learning feasible. We exploit customer review websites to crawl a large set of online shopping images and corresponding offline user photos with fine-grained clothing attributes, i.e., around 450,000 online shopping images and about 90,000 exact offline counterpart images of those online ones. All these images are collected from real-world consumer websites reflecting the diversity of the data modality, which makes this dataset unique and rare in the academic community. We extensively evaluate the retrieval performance of networks in different configurations. The top-20 retrieval accuracy is doubled when using the proposed DARN other than the current popular solution using pre-trained CNN features only (0.570 vs. 0.268).