More complex encoder is not all you need

TL;DR

This paper introduces neU-Net, a novel medical image segmentation model that emphasizes a powerful decoder with sub-pixel convolution and multi-scale wavelet inputs, achieving superior results over existing methods.

Contribution

The paper proposes a decoder-focused U-Net variant with sub-pixel convolution and multi-scale wavelet inputs, addressing limitations of traditional encoder-centric designs.

Findings

Outperforms state-of-the-art on Synapse dataset

Achieves superior results on ACDC dataset

Highlights importance of decoder design in segmentation

Abstract

U-Net and its variants have been widely used in medical image segmentation. However, most current U-Net variants confine their improvement strategies to building more complex encoder, while leaving the decoder unchanged or adopting a simple symmetric structure. These approaches overlook the true functionality of the decoder: receiving low-resolution feature maps from the encoder and restoring feature map resolution and lost information through upsampling. As a result, the decoder, especially its upsampling component, plays a crucial role in enhancing segmentation outcomes. However, in 3D medical image segmentation, the commonly used transposed convolution can result in visual artifacts. This issue stems from the absence of direct relationship between adjacent pixels in the output feature map. Furthermore, plain encoder has already possessed sufficient feature extraction capability because downsampling operation leads to the gradual expansion of the receptive field, but the loss of information during downsampling process is unignorable. To address the gap in relevant research, we extend our focus beyond the encoder and introduce neU-Net (i.e., not complex encoder U-Net), which incorporates a novel Sub-pixel Convolution for upsampling to construct a powerful decoder. Additionally, we introduce multi-scale wavelet inputs module on the encoder side to provide additional information. Our model design achieves excellent results, surpassing other state-of-the-art methods on both the Synapse and ACDC datasets.