GKGNet: Group K-Nearest Neighbor based Graph Convolutional Network for Multi-Label Image Recognition

TL;DR

GKGNet is a novel graph convolutional network that models label-image relationships for multi-label image recognition, achieving state-of-the-art results with lower computational costs by dynamically constructing graphs.

Contribution

This work introduces the first fully graph-based model for MLIR, utilizing a Group KGCN module for flexible, multi-perspective graph construction and message passing.

Findings

Achieves state-of-the-art performance on MS-COCO and VOC2007 datasets.

Reduces computational costs compared to existing methods.

Effectively models complex label-image relationships.

Abstract

Multi-Label Image Recognition (MLIR) is a challenging task that aims to predict multiple object labels in a single image while modeling the complex relationships between labels and image regions. Although convolutional neural networks and vision transformers have succeeded in processing images as regular grids of pixels or patches, these representations are sub-optimal for capturing irregular and discontinuous regions of interest. In this work, we present the first fully graph convolutional model, Group K-nearest neighbor based Graph convolutional Network (GKGNet), which models the connections between semantic label embeddings and image patches in a flexible and unified graph structure. To address the scale variance of different objects and to capture information from multiple perspectives, we propose the Group KGCN module for dynamic graph construction and message passing. Our experiments demonstrate that GKGNet achieves state-of-the-art performance with significantly lower computational costs on the challenging multi-label datasets, i.e., MS-COCO and VOC2007 datasets. Codes are available at https://github.com/jin-s13/GKGNet.