Visual Search at Alibaba

TL;DR

This paper presents Alibaba's large-scale visual search system, addressing challenges like heterogeneous data, large-scale indexing, and effective deep model training, using deep learning and system optimization for real-world commercial deployment.

Contribution

The paper introduces a comprehensive visual search system at Alibaba, including novel deep CNN models, fusion techniques, and scalable indexing methods tailored for large-scale e-commerce applications.

Findings

Effective category prediction through model and search fusion.

Deep CNN for joint detection and feature learning from user clicks.

Scalable binary indexing maintaining high recall and precision.

Abstract

This paper introduces the large scale visual search algorithm and system infrastructure at Alibaba. The following challenges are discussed under the E-commercial circumstance at Alibaba (a) how to handle heterogeneous image data and bridge the gap between real-shot images from user query and the online images. (b) how to deal with large scale indexing for massive updating data. (c) how to train deep models for effective feature representation without huge human annotations. (d) how to improve the user engagement by considering the quality of the content. We take advantage of large image collection of Alibaba and state-of-the-art deep learning techniques to perform visual search at scale. We present solutions and implementation details to overcome those problems and also share our learnings from building such a large scale commercial visual search engine. Specifically, model and search-based fusion approach is introduced to effectively predict categories. Also, we propose a deep CNN model for joint detection and feature learning by mining user click behavior. The binary index engine is designed to scale up indexing without compromising recall and precision. Finally, we apply all the stages into an end-to-end system architecture, which can simultaneously achieve highly efficient and scalable performance adapting to real-shot images. Extensive experiments demonstrate the advancement of each module in our system. We hope visual search at Alibaba becomes more widely incorporated into today's commercial applications.