A Multihead Continual Learning Framework for Fine-Grained Fashion Image Retrieval with Contrastive Learning and Exponential Moving Average Distillation

TL;DR

This paper introduces a multihead continual learning framework for fine-grained fashion image retrieval that effectively handles evolving classes, improves efficiency, and maintains high accuracy through contrastive learning and EMA distillation.

Contribution

It presents a novel multihead continual learning approach with contrastive learning and EMA distillation tailored for dynamic fine-grained fashion image retrieval tasks.

Findings

Outperforms existing CIL baselines with similar training costs.

Achieves comparable accuracy to static methods using only 30% of the training cost.

Demonstrates scalability and efficiency across four datasets.

Abstract

Most fine-grained fashion image retrieval (FIR) methods assume a static setting, requiring full retraining when new attributes appear, which is costly and impractical for dynamic scenarios. Although pretrained models support zero-shot inference, their accuracy drops without supervision, and no prior work explores class-incremental learning (CIL) for fine-grained FIR. We propose a multihead continual learning framework for fine-grained fashion image retrieval with contrastive learning and exponential moving average (EMA) distillation (MCL-FIR). MCL-FIR adopts a multi-head design to accommodate evolving classes across increments, reformulates triplet inputs into doublets with InfoNCE for simpler and more effective training, and employs EMA distillation for efficient knowledge transfer. Experiments across four datasets demonstrate that, beyond its scalability, MCL-FIR achieves a strong balance between efficiency and accuracy. It significantly outperforms CIL baselines under similar training cost, and compared with static methods, it delivers comparable performance while using only about 30% of the training cost. The source code is publicly available in https://github.com/Dr-LingXiao/MCL-FIR.