Multi-Scale Relational Graph Convolutional Network for Multiple Instance Learning in Histopathology Images

TL;DR

This paper introduces MS-RGCN, a multi-scale relational graph convolutional network for histopathology image analysis that effectively integrates multi-magnification information for improved cancer grading.

Contribution

The paper presents a novel multi-scale relational graph convolutional network that handles different embedding spaces at each magnification for early fusion in histopathology image analysis.

Findings

Outperforms state-of-the-art methods on multiple datasets

Effective integration of multi-magnification information

Demonstrates the importance of early fusion in histopathology analysis

Abstract

Graph convolutional neural networks have shown significant potential in natural and histopathology images. However, their use has only been studied in a single magnification or multi-magnification with late fusion. In order to leverage the multi-magnification information and early fusion with graph convolutional networks, we handle different embedding spaces at each magnification by introducing the Multi-Scale Relational Graph Convolutional Network (MS-RGCN) as a multiple instance learning method. We model histopathology image patches and their relation with neighboring patches and patches at other scales (i.e., magnifications) as a graph. To pass the information between different magnification embedding spaces, we define separate message-passing neural networks based on the node and edge type. We experiment on prostate cancer histopathology images to predict the grade groups based on the extracted features from patches. We also compare our MS-RGCN with multiple state-of-the-art methods with evaluations on several source and held-out datasets. Our method outperforms the state-of-the-art on all of the datasets and image types consisting of tissue microarrays, whole-mount slide regions, and whole-slide images. Through an ablation study, we test and show the value of the pertinent design features of the MS-RGCN.