Dynamic Graph Neural Network with Adaptive Features Selection for RGB-D Based Indoor Scene Recognition

TL;DR

This paper introduces a dynamic graph neural network with adaptive feature selection for RGB-D indoor scene recognition, effectively modeling object relations and selecting key local features to improve accuracy.

Contribution

It proposes a novel adaptive node selection mechanism within a dynamic graph model to better exploit local features from RGB and depth modalities for scene recognition.

Findings

Outperforms state-of-the-art methods on SUN RGB-D and NYU Depth v2 datasets.

Effectively models object relations at three different levels.

Demonstrates superior recognition accuracy through adaptive feature exploitation.

Abstract

Multi-modality of color and depth, i.e., RGB-D, is of great importance in recent research of indoor scene recognition. In this kind of data representation, depth map is able to describe the 3D structure of scenes and geometric relations among objects. Previous works showed that local features of both modalities are vital for promotion of recognition accuracy. However, the problem of adaptive selection and effective exploitation on these key local features remains open in this field. In this paper, a dynamic graph model is proposed with adaptive node selection mechanism to solve the above problem. In this model, a dynamic graph is built up to model the relations among objects and scene, and a method of adaptive node selection is proposed to take key local features from both modalities of RGB and depth for graph modeling. After that, these nodes are grouped by three different levels, representing near or far relations among objects. Moreover, the graph model is updated dynamically according to attention weights. Finally, the updated and optimized features of RGB and depth modalities are fused together for indoor scene recognition. Experiments are performed on public datasets including SUN RGB-D and NYU Depth v2. Extensive results demonstrate that our method has superior performance when comparing to state-of-the-arts methods, and show that the proposed method is able to exploit crucial local features from both modalities of RGB and depth.