Multi-Label Image Recognition with Graph Convolutional Networks

TL;DR

This paper introduces a graph convolutional network-based model for multi-label image recognition that captures label dependencies to improve accuracy, demonstrating superior performance and meaningful semantic topology in classifiers.

Contribution

The paper proposes a novel GCN-based approach with a re-weighted label correlation scheme for end-to-end multi-label image recognition.

Findings

Outperforms existing state-of-the-art methods on benchmark datasets.

Effectively models label dependencies through GCN.

Maintains meaningful semantic topology in learned classifiers.

Abstract

The task of multi-label image recognition is to predict a set of object labels that present in an image. As objects normally co-occur in an image, it is desirable to model the label dependencies to improve the recognition performance. To capture and explore such important dependencies, we propose a multi-label classification model based on Graph Convolutional Network (GCN). The model builds a directed graph over the object labels, where each node (label) is represented by word embeddings of a label, and GCN is learned to map this label graph into a set of inter-dependent object classifiers. These classifiers are applied to the image descriptors extracted by another sub-net, enabling the whole network to be end-to-end trainable. Furthermore, we propose a novel re-weighted scheme to create an effective label correlation matrix to guide information propagation among the nodes in GCN. Experiments on two multi-label image recognition datasets show that our approach obviously outperforms other existing state-of-the-art methods. In addition, visualization analyses reveal that the classifiers learned by our model maintain meaningful semantic topology.