Decoupling Endpoint and Semantic Transition Learning for Zero-Shot Composed Image Retrieval

TL;DR

This paper introduces DeCIR, a method that decouples endpoint and semantic transition learning in zero-shot composed image retrieval, significantly improving performance without added inference complexity.

Contribution

DeCIR proposes a novel decoupling approach with separate training of endpoint and transition modules, addressing semantic transition bottlenecks in projection-based ZS-CIR.

Findings

DeCIR consistently outperforms existing projection-based ZS-CIR methods.

The method improves retrieval accuracy on multiple datasets.

DeCIR maintains low inference complexity while enhancing performance.

Abstract

Zero-shot composed image retrieval (ZS-CIR) retrieves a target image from a reference image and a text modification without human-annotated CIR triplets. Projection-based ZS-CIR methods are attractive because they do not rely on LLMs at inference and remain lightweight, but they often underperform LLM-based approaches on complex semantic modifications. This gap reflects a semantic transition bottleneck in projection-based ZS-CIR: endpoint-level matching can let the edit text act as a target-side attribute cue rather than grounding it as a source-conditioned semantic transition. We further show that adding semantic transition supervision to the same text adapter creates an endpoint--transition conflict between endpoint alignment and semantic transition alignment. To address this conflict, DeCIR decouples endpoint and transition learning. It constructs paired forward/reverse edit tuples from image-caption pairs, trains separate low-rank text adapter branches for endpoint alignment and semantic transition alignment, and merges them with Low-Rank Directional Merge (LRDM) into one deployable adapter. Extensive experiments on CIRR, CIRCO, FashionIQ, and GeneCIS demonstrate that DeCIR consistently improves projection-based ZS-CIR without increasing inference complexity.