SegNetr: Rethinking the local-global interactions and skip connections in U-shaped networks

TL;DR

SegNetr introduces a lightweight U-shaped network with dynamic local-global interaction blocks and spatially aware skip connections, achieving high performance with fewer parameters and GFLOPs in medical image segmentation.

Contribution

The paper proposes a novel SegNetr block for dynamic local-global interactions and an information retention skip connection, enhancing U-shaped networks' efficiency and accuracy.

Findings

Fewer parameters and GFLOPs than vanilla U-Net

Achieves comparable segmentation performance to state-of-the-art methods

Applicable components to other U-shaped networks

Abstract

Recently, U-shaped networks have dominated the field of medical image segmentation due to their simple and easily tuned structure. However, existing U-shaped segmentation networks: 1) mostly focus on designing complex self-attention modules to compensate for the lack of long-term dependence based on convolution operation, which increases the overall number of parameters and computational complexity of the network; 2) simply fuse the features of encoder and decoder, ignoring the connection between their spatial locations. In this paper, we rethink the above problem and build a lightweight medical image segmentation network, called SegNetr. Specifically, we introduce a novel SegNetr block that can perform local-global interactions dynamically at any stage and with only linear complexity. At the same time, we design a general information retention skip connection (IRSC) to preserve the spatial location information of encoder features and achieve accurate fusion with the decoder features. We validate the effectiveness of SegNetr on four mainstream medical image segmentation datasets, with 59\% and 76\% fewer parameters and GFLOPs than vanilla U-Net, while achieving segmentation performance comparable to state-of-the-art methods. Notably, the components proposed in this paper can be applied to other U-shaped networks to improve their segmentation performance.