ACC-UNet: A Completely Convolutional UNet model for the 2020s

TL;DR

ACC-UNet is a fully convolutional neural network that emulates transformer advantages like long-range dependencies, achieving state-of-the-art medical image segmentation performance with fewer parameters.

Contribution

The paper introduces ACC-UNet, a novel convolutional UNet that incorporates transformer-inspired features without using transformers, outperforming existing models.

Findings

Outperforms state-of-the-art models Swin-Unet and UCTransNet in dice score.

Uses significantly fewer parameters than competing models.

Consistently outperforms convnets, transformers, and hybrids across five benchmarks.

Abstract

This decade is marked by the introduction of Vision Transformer, a radical paradigm shift in broad computer vision. A similar trend is followed in medical imaging, UNet, one of the most influential architectures, has been redesigned with transformers. Recently, the efficacy of convolutional models in vision is being reinvestigated by seminal works such as ConvNext, which elevates a ResNet to Swin Transformer level. Deriving inspiration from this, we aim to improve a purely convolutional UNet model so that it can be on par with the transformer-based models, e.g, Swin-Unet or UCTransNet. We examined several advantages of the transformer-based UNet models, primarily long-range dependencies and cross-level skip connections. We attempted to emulate them through convolution operations and thus propose, ACC-UNet, a completely convolutional UNet model that brings the best of both worlds, the inherent inductive biases of convnets with the design decisions of transformers. ACC-UNet was evaluated on 5 different medical image segmentation benchmarks and consistently outperformed convnets, transformers, and their hybrids. Notably, ACC-UNet outperforms state-of-the-art models Swin-Unet and UCTransNet by $2.64 \pm 2.54\%$ and $0.45 \pm 1.61\%$ in terms of dice score, respectively, while using a fraction of their parameters ($59.26\%$ and $24.24\%$). Our codes are available at https://github.com/kiharalab/ACC-UNet.