Web-Scale Responsive Visual Search at Bing

TL;DR

This paper presents a scalable, fast visual search system for Bing that handles billions of images with deep learning features, achieving sub-200ms response times through a cascaded learning-to-rank framework on a distributed platform.

Contribution

The paper introduces a novel large-scale visual search system with a cascaded deep learning ranking framework deployed at web scale in Bing.

Findings

Supports tens of billions of images with fast response times

Employs a cascaded learning-to-rank framework for relevance

Demonstrates effectiveness through experiments

Abstract

In this paper, we introduce a web-scale general visual search system deployed in Microsoft Bing. The system accommodates tens of billions of images in the index, with thousands of features for each image, and can respond in less than 200 ms. In order to overcome the challenges in relevance, latency, and scalability in such large scale of data, we employ a cascaded learning-to-rank framework based on various latest deep learning visual features, and deploy in a distributed heterogeneous computing platform. Quantitative and qualitative experiments show that our system is able to support various applications on Bing website and apps.