Give me a hint! Navigating Image Databases using Human-in-the-loop Feedback

TL;DR

This paper presents an interactive image search method that uses human feedback and deep reinforcement learning to improve search results without relying on attribute annotations, enhancing user experience and search accuracy.

Contribution

It introduces a reinforcement learning-based image selection strategy and extends similarity networks to improve interactive image search and attribute representation.

Findings

Effective in refining search results through human-in-the-loop feedback

Deep reinforcement model outperforms hand-crafted image selection measures

Improved image attribute embeddings with global similarity incorporation

Abstract

In this paper, we introduce an attribute-based interactive image search which can leverage human-in-the-loop feedback to iteratively refine image search results. We study active image search where human feedback is solicited exclusively in visual form, without using relative attribute annotations used by prior work which are not typically found in many datasets. In order to optimize the image selection strategy, a deep reinforcement model is trained to learn what images are informative rather than rely on hand-crafted measures typically leveraged in prior work. Additionally, we extend the recently introduced Conditional Similarity Network to incorporate global similarity in training visual embeddings, which results in more natural transitions as the user explores the learned similarity embeddings. Our experiments demonstrate the effectiveness of our approach, producing compelling results on both active image search and image attribute representation tasks.