Semantic Contrastive Bootstrapping for Single-positive Multi-label Recognition

TL;DR

This paper introduces a novel semantic contrastive bootstrapping method for single-positive multi-label image recognition, improving label relationship recovery and achieving state-of-the-art results on multiple benchmarks.

Contribution

It proposes a new bootstrapping framework with semantic contrastive learning and a recurrent transformer to enhance multi-label recognition with limited annotations.

Findings

Outperforms existing models on four benchmarks

Effectively recovers cross-object relationships

Reduces annotation effort needed for training

Abstract

Learning multi-label image recognition with incomplete annotation is gaining popularity due to its superior performance and significant labor savings when compared to training with fully labeled datasets. Existing literature mainly focuses on label completion and co-occurrence learning while facing difficulties with the most common single-positive label manner. To tackle this problem, we present a semantic contrastive bootstrapping (Scob) approach to gradually recover the cross-object relationships by introducing class activation as semantic guidance. With this learning guidance, we then propose a recurrent semantic masked transformer to extract iconic object-level representations and delve into the contrastive learning problems on multi-label classification tasks. We further propose a bootstrapping framework in an Expectation-Maximization fashion that iteratively optimizes the network parameters and refines semantic guidance to alleviate possible disturbance caused by wrong semantic guidance. Extensive experimental results demonstrate that the proposed joint learning framework surpasses the state-of-the-art models by a large margin on four public multi-label image recognition benchmarks. Codes can be found at https://github.com/iCVTEAM/Scob.