FNOSeg3D: Resolution-Robust 3D Image Segmentation with Fourier Neural Operator

TL;DR

FNOSeg3D introduces a resolution-robust 3D segmentation model based on Fourier neural operators, achieving high accuracy and efficiency in medical image segmentation across varying resolutions.

Contribution

The paper presents FNOSeg3D, a novel 3D segmentation model that leverages Fourier neural operators with improvements for resolution robustness and parameter efficiency.

Findings

Outperforms other models in resolution robustness on BraTS'19 dataset

Uses less than 1% of parameters compared to competing models

Achieves superior accuracy across different training resolutions

Abstract

Due to the computational complexity of 3D medical image segmentation, training with downsampled images is a common remedy for out-of-memory errors in deep learning. Nevertheless, as standard spatial convolution is sensitive to variations in image resolution, the accuracy of a convolutional neural network trained with downsampled images can be suboptimal when applied on the original resolution. To address this limitation, we introduce FNOSeg3D, a 3D segmentation model robust to training image resolution based on the Fourier neural operator (FNO). The FNO is a deep learning framework for learning mappings between functions in partial differential equations, which has the appealing properties of zero-shot super-resolution and global receptive field. We improve the FNO by reducing its parameter requirement and enhancing its learning capability through residual connections and deep supervision, and these result in our FNOSeg3D model which is parameter efficient and resolution robust. When tested on the BraTS'19 dataset, it achieved superior robustness to training image resolution than other tested models with less than 1% of their model parameters.