OFFSET: Segmentation-based Focus Shift Revision for Composed Image Retrieval
Zhiwei Chen, Yupeng Hu, Zixu Li, Zhiheng Fu, Xuemeng Song, Liqiang Nie

TL;DR
OFFSET introduces a segmentation-based focus shift revision method for composed image retrieval, effectively addressing noise and focus bias issues to improve retrieval accuracy using a novel focus mapping approach.
Contribution
The paper proposes a new focus mapping-based feature extractor and a textually guided focus revision module to enhance composed image retrieval performance.
Findings
Outperforms existing methods on four benchmark datasets.
Effectively reduces noise interference in visual features.
Improves focus alignment between text and image modifications.
Abstract
Composed Image Retrieval (CIR) represents a novel retrieval paradigm that is capable of expressing users' intricate retrieval requirements flexibly. It enables the user to give a multimodal query, comprising a reference image and a modification text, and subsequently retrieve the target image. Notwithstanding the considerable advances made by prevailing methodologies, CIR remains in its nascent stages due to two limitations: 1) inhomogeneity between dominant and noisy portions in visual data is ignored, leading to query feature degradation, and 2) the priority of textual data in the image modification process is overlooked, which leads to a visual focus bias. To address these two limitations, this work presents a focus mapping-based feature extractor, which consists of two modules: dominant portion segmentation and dual focus mapping. It is designed to identify significant dominant…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMultimodal Machine Learning Applications · Image Retrieval and Classification Techniques · Visual Attention and Saliency Detection
