Dynamic Graph Message Passing Networks

TL;DR

This paper introduces a dynamic graph message passing network that adaptively samples nodes for efficient long-range dependency modeling in scene understanding, outperforming state-of-the-art methods with fewer computations.

Contribution

It proposes a novel adaptive sampling method for graph message passing that reduces computational costs while maintaining high performance.

Findings

Significant improvements over strong baselines on three tasks

Outperforms fully-connected graphs with fewer parameters

Reduces floating-point operations compared to related models

Abstract

Modelling long-range dependencies is critical for scene understanding tasks in computer vision. Although CNNs have excelled in many vision tasks, they are still limited in capturing long-range structured relationships as they typically consist of layers of local kernels. A fully-connected graph is beneficial for such modelling, however, its computational overhead is prohibitive. We propose a dynamic graph message passing network, that significantly reduces the computational complexity compared to related works modelling a fully-connected graph. This is achieved by adaptively sampling nodes in the graph, conditioned on the input, for message passing. Based on the sampled nodes, we dynamically predict node-dependent filter weights and the affinity matrix for propagating information between them. Using this model, we show significant improvements with respect to strong, state-of-the-art baselines on three different tasks and backbone architectures. Our approach also outperforms fully-connected graphs while using substantially fewer floating-point operations and parameters. The project website is http://www.robots.ox.ac.uk/~lz/dgmn/