Semantic-Aware Representation Learning via Conditional Transport for Multi-Label Image Classification

TL;DR

This paper introduces SCT, a novel framework that enhances multi-label image classification by learning semantic-aware features and achieving fine-grained visual-semantic alignment, leading to improved performance.

Contribution

It proposes a unified approach combining semantic-related feature learning and conditional transport-based alignment for better multi-label classification.

Findings

SCT outperforms existing methods on VOC2007 and MS-COCO datasets.

The semantic-related feature module improves discriminative label-specific features.

Conditional transport enables precise visual-semantic alignment.

Abstract

Multi-label image classification is a critical task in machine learning that aims to accurately assign multiple labels to a single image. While existing methods often utilize attention mechanisms or graph convolutional networks to model visual representations, their performance is still constrained by two critical limitations: the inability to learn discriminative semantic-aware features, and the lack of fine-grained alignment between visual representations and label embeddings. To tackle these issues in a unified framework, this paper proposes a novel approach named Semantic-aware representation learning via Conditional Transport for Multi-Label Image Classification (SCT). The proposed method introduces a semantic-related feature learning module that extracts discriminative label-specific features by emphasizing semantic relevance and interaction, along with a conditional transport-based alignment mechanism that enables precise visual-semantic alignment. Extensive experiments on two widely-used benchmark datasets, VOC2007 and MS-COCO, validate the effectiveness of SCT and demonstrate its superior performance compared to existing state-of-the-art methods.