Learning Semantic-Aware Threshold for Multi-Label Image Recognition with Partial Labels

TL;DR

This paper proposes a novel Semantic-Aware Threshold Learning (SATL) method for multi-label image recognition with partial labels, dynamically adapting thresholds based on score distributions to improve pseudo-label accuracy and model performance.

Contribution

Introduces SATL, which calculates and updates category-specific thresholds using score distributions, enhancing pseudo-label quality in partial label scenarios.

Findings

Significant performance improvements on COCO and VG-200 datasets.

Effective dynamic threshold adjustment based on score distributions.

Enhanced discrimination between positive and negative samples.

Abstract

Multi-label image recognition with partial labels (MLR-PL) is designed to train models using a mix of known and unknown labels. Traditional methods rely on semantic or feature correlations to create pseudo-labels for unidentified labels using pre-set thresholds. This approach often overlooks the varying score distributions across categories, resulting in inaccurate and incomplete pseudo-labels, thereby affecting performance. In our study, we introduce the Semantic-Aware Threshold Learning (SATL) algorithm. This innovative approach calculates the score distribution for both positive and negative samples within each category and determines category-specific thresholds based on these distributions. These distributions and thresholds are dynamically updated throughout the learning process. Additionally, we implement a differential ranking loss to establish a significant gap between the score distributions of positive and negative samples, enhancing the discrimination of the thresholds. Comprehensive experiments and analysis on large-scale multi-label datasets, such as Microsoft COCO and VG-200, demonstrate that our method significantly improves performance in scenarios with limited labels.