Semantic Representation and Dependency Learning for Multi-Label Image Recognition

TL;DR

This paper introduces a novel semantic representation and dependency learning framework for multi-label image recognition that avoids reliance on pre-trained object detection models and improves recognition of rare categories.

Contribution

The proposed SRDL framework learns category-specific semantic features and dependencies without pre-trained detection models, enhancing multi-label recognition performance.

Findings

Outperforms state-of-the-art on MS-COCO and Pascal VOC 2007 datasets.

Effectively captures semantic dependencies among categories.

Improves recognition accuracy for rare categories.

Abstract

Recently many multi-label image recognition (MLR) works have made significant progress by introducing pre-trained object detection models to generate lots of proposals or utilizing statistical label co-occurrence enhance the correlation among different categories. However, these works have some limitations: (1) the effectiveness of the network significantly depends on pre-trained object detection models that bring expensive and unaffordable computation; (2) the network performance degrades when there exist occasional co-occurrence objects in images, especially for the rare categories. To address these problems, we propose a novel and effective semantic representation and dependency learning (SRDL) framework to learn category-specific semantic representation for each category and capture semantic dependency among all categories. Specifically, we design a category-specific attentional regions (CAR) module to generate channel/spatial-wise attention matrices to guide model to focus on semantic-aware regions. We also design an object erasing (OE) module to implicitly learn semantic dependency among categories by erasing semantic-aware regions to regularize the network training. Extensive experiments and comparisons on two popular MLR benchmark datasets (i.e., MS-COCO and Pascal VOC 2007) demonstrate the effectiveness of the proposed framework over current state-of-the-art algorithms.