Query-free Clothing Retrieval via Implicit Relevance Feedback

TL;DR

This paper introduces a novel clothing retrieval method that uses implicit relevance feedback and Bayesian modeling to find clothing items based on users' mental images without explicit queries.

Contribution

It proposes a new approach for image retrieval that models user intent through relevance feedback and Bayesian inference, addressing the challenge of mental image retrieval in online shopping.

Findings

Effective retrieval with only user clicks as feedback

Bayesian model captures variability in human decision-making

Demonstrated success with real user experiments

Abstract

Image-based clothing retrieval is receiving increasing interest with the growth of online shopping. In practice, users may often have a desired piece of clothing in mind (e.g., either having seen it before on the street or requiring certain specific clothing attributes) but may be unable to supply an image as a query. We model this problem as a new type of image retrieval task in which the target image resides only in the user's mind (called "mental image retrieval" hereafter). Because of the absence of an explicit query image, we propose to solve this problem through relevance feedback. Specifically, a new Bayesian formulation is proposed that simultaneously models the retrieval target and its high-level representation in the mind of the user (called the "user metric" hereafter) as posterior distributions of pre-fetched shop images and heterogeneous features extracted from multiple clothing attributes, respectively. Requiring only clicks as user feedback, the proposed algorithm is able to account for the variability in human decision-making. Experiments with real users demonstrate the effectiveness of the proposed algorithm.