TomoGraphView: 3D Medical Image Classification with Omnidirectional Slice Representations and Graph Neural Networks

TL;DR

TomoGraphView introduces a novel 3D medical image classification approach that uses omnidirectional slicing and graph neural networks to better capture spatial structures in volumetric data, overcoming limitations of traditional slice-based methods.

Contribution

The paper presents a new framework combining omnidirectional volume slicing with spherical graph aggregation, enhancing 3D feature extraction beyond standard axial, sagittal, and coronal views.

Findings

Improved accuracy in tumor classification tasks.

Effective capture of complex 3D spatial dependencies.

Open-source code and tools for omnidirectional slicing.

Abstract

The sharp rise in medical tomography examinations has created a demand for automated systems that can reliably extract informative features for downstream tasks such as tumor characterization. Although 3D volumes contain richer information than individual slices, effective 3D classification remains difficult: volumetric data encode complex spatial dependencies, and the scarcity of large-scale 3D datasets has constrained progress toward 3D foundation models. As a result, many recent approaches rely on 2D vision foundation models trained on natural images, repurposing them as feature extractors for medical scans with surprisingly strong performance. Despite their practical success, current methods that apply 2D foundation models to 3D scans via slice-based decomposition remain fundamentally limited. Standard slicing along axial, sagittal, and coronal planes often fails to capture the true spatial extent of a structure when its orientation does not align with these canonical views. More critically, most approaches aggregate slice features independently, ignoring the underlying 3D geometry and losing spatial coherence across slices. To overcome these limitations, we propose TomoGraphView, a novel framework that integrates omnidirectional volume slicing with spherical graph-based feature aggregation. Instead of restricting the model to axial, sagittal, or coronal planes, our method samples both canonical and non-canonical cross-sections generated from uniformly distributed points on a sphere enclosing the volume. We publicly share our accessible code base at http://github.com/compai-lab/2025-MedIA-kiechle and provide a user-friendly library for omnidirectional volume slicing at https://pypi.org/project/OmniSlicer.