Transductive CLIP with Class-Conditional Contrastive Learning

TL;DR

This paper introduces Transductive CLIP, a framework that leverages CLIP's zero-shot capabilities and class-conditional contrastive learning to effectively train classifiers with noisy labels, improving robustness and accuracy.

Contribution

It presents a novel class-conditional contrastive learning method combined with ensemble pseudo-label updating to handle noisy supervision from CLIP.

Findings

Significant performance improvements over state-of-the-art methods.

Effective mitigation of label noise impact in classification tasks.

Robust training with noisy labels demonstrated on benchmark datasets.

Abstract

Inspired by the remarkable zero-shot generalization capacity of vision-language pre-trained model, we seek to leverage the supervision from CLIP model to alleviate the burden of data labeling. However, such supervision inevitably contains the label noise, which significantly degrades the discriminative power of the classification model. In this work, we propose Transductive CLIP, a novel framework for learning a classification network with noisy labels from scratch. Firstly, a class-conditional contrastive learning mechanism is proposed to mitigate the reliance on pseudo labels and boost the tolerance to noisy labels. Secondly, ensemble labels is adopted as a pseudo label updating strategy to stabilize the training of deep neural networks with noisy labels. This framework can reduce the impact of noisy labels from CLIP model effectively by combining both techniques. Experiments on multiple benchmark datasets demonstrate the substantial improvements over other state-of-the-art methods.