Super-Resolution Based Patch-Free 3D Image Segmentation with High-Frequency Guidance

TL;DR

This paper introduces a super-resolution based patch-free 3D image segmentation framework that achieves high-resolution segmentation efficiently by combining global low-resolution input with auxiliary super-resolution tasks and high-frequency guidance.

Contribution

It proposes a novel patch-free framework with high-frequency guidance and task fusion, improving inference speed and segmentation accuracy over existing patch-based and patch-free methods.

Findings

Four times faster inference than patch-based methods

Outperforms existing patch-based and patch-free models in accuracy

Effective reconstruction of high-frequency details from low-resolution input

Abstract

High resolution (HR) 3D images are widely used nowadays, such as medical images like Magnetic Resonance Imaging (MRI) and Computed Tomography (CT). However, segmentation of these 3D images remains a challenge due to their high spatial resolution and dimensionality in contrast to currently limited GPU memory. Therefore, most existing 3D image segmentation methods use patch-based models, which have low inference efficiency and ignore global contextual information. To address these problems, we propose a super-resolution (SR) based patch-free 3D image segmentation framework that can realize HR segmentation from a global-wise low-resolution (LR) input. The framework contains two sub-tasks, of which semantic segmentation is the main task and super resolution is an auxiliary task aiding in rebuilding the high frequency information from the LR input. To furthermore balance the information loss with the LR input, we propose a High-Frequency Guidance Module (HGM), and design an efficient selective cropping algorithm to crop an HR patch from the original image as restoration guidance for it. In addition, we also propose a Task-Fusion Module (TFM) to exploit the inter connections between segmentation and SR task, realizing joint optimization of the two tasks. When predicting, only the main segmentation task is needed, while other modules can be removed for acceleration. The experimental results on two different datasets show that our framework has a four times higher inference speed compared to traditional patch-based methods, while its performance also surpasses other patch-based and patch-free models.