EurNet: Efficient Multi-Range Relational Modeling of Spatial Multi-Relational Data

TL;DR

EurNet introduces a multi-range relational modeling approach that constructs a graph with different spatial relation types and uses a novel message passing layer, achieving state-of-the-art results in image and protein data tasks.

Contribution

EurNet's novel multi-relational graph construction and gated relational message passing layer effectively model diverse spatial relations with minimal extra computation.

Findings

Outperforms FocalNet on ImageNet, COCO, and ADE20K.

Surpasses GearNet on protein function prediction benchmarks.

Demonstrates strong generalization across image and protein domains.

Abstract

Modeling spatial relationship in the data remains critical across many different tasks, such as image classification, semantic segmentation and protein structure understanding. Previous works often use a unified solution like relative positional encoding. However, there exists different kinds of spatial relations, including short-range, medium-range and long-range relations, and modeling them separately can better capture the focus of different tasks on the multi-range relations (e.g., short-range relations can be important in instance segmentation, while long-range relations should be upweighted for semantic segmentation). In this work, we introduce the EurNet for Efficient multi-range relational modeling. EurNet constructs the multi-relational graph, where each type of edge corresponds to short-, medium- or long-range spatial interactions. In the constructed graph, EurNet adopts a novel modeling layer, called gated relational message passing (GRMP), to propagate multi-relational information across the data. GRMP captures multiple relations within the data with little extra computational cost. We study EurNets in two important domains for image and protein structure modeling. Extensive experiments on ImageNet classification, COCO object detection and ADE20K semantic segmentation verify the gains of EurNet over the previous SoTA FocalNet. On the EC and GO protein function prediction benchmarks, EurNet consistently surpasses the previous SoTA GearNet. Our results demonstrate the strength of EurNets on modeling spatial multi-relational data from various domains. The implementations of EurNet for image modeling are available at https://github.com/hirl-team/EurNet-Image . The implementations for other applied domains/tasks will be released soon.