Robust Relevance Feedback for Interactive Known-Item Video Search

TL;DR

This paper introduces a robust relevance feedback method for interactive known-item video search, improving search accuracy despite inconsistent user feedback by modeling multiple user perceptions and filtering misaligned feedback.

Contribution

It proposes a novel approach combining pairwise relative judgment feedback and multiple sub-perception embeddings to enhance robustness in known-item video search.

Findings

Achieves over 60% success in ranking targets to top positions from initial ranks 10-50.

Attains over 40% success rate for targets initially ranked between 1,000 and 5,000.

Demonstrates robustness of relevance feedback despite inconsistent user judgments.

Abstract

Known-item search (KIS) involves only a single search target, making relevance feedback-typically a powerful technique for efficiently identifying multiple positive examples to infer user intent-inapplicable. PicHunter addresses this issue by asking users to select the top-k most similar examples to the unique search target from a displayed set. Under ideal conditions, when the user's perception aligns closely with the machine's perception of similarity, consistent and precise judgments can elevate the target to the top position within a few iterations. However, in practical scenarios, expecting users to provide consistent judgments is often unrealistic, especially when the underlying embedding features used for similarity measurements lack interpretability. To enhance robustness, we first introduce a pairwise relative judgment feedback that improves the stability of top-k selections by mitigating the impact of misaligned feedback. Then, we decompose user perception into multiple sub-perceptions, each represented as an independent embedding space. This approach assumes that users may not consistently align with a single representation but are more likely to align with one or several among multiple representations. We develop a predictive user model that estimates the combination of sub-perceptions based on each user feedback instance. The predictive user model is then trained to filter out the misaligned sub-perceptions. Experimental evaluations on the large-scale open-domain dataset V3C indicate that the proposed model can optimize over 60% search targets to the top rank when their initial ranks at the search depth between 10 and 50. Even for targets initially ranked between 1,000 and 5,000, the model achieves a success rate exceeding 40% in optimizing ranks to the top, demonstrating the enhanced robustness of relevance feedback in KIS despite inconsistent feedback.