Full-resolution MLPs Empower Medical Dense Prediction

TL;DR

This paper introduces a full-resolution hierarchical MLP framework for medical dense prediction tasks, demonstrating that MLPs outperform CNNs and transformers at full image resolution across various applications.

Contribution

The study proposes a novel full-resolution hierarchical MLP framework that effectively captures tissue-level details, surpassing CNN and transformer models in medical dense prediction.

Findings

Outperforms CNNs and transformers on multiple datasets

Achieves state-of-the-art results in medical image restoration, registration, and segmentation

Validates the effectiveness of full-resolution MLPs in medical vision tasks

Abstract

Dense prediction is a fundamental requirement for many medical vision tasks such as medical image restoration, registration, and segmentation. The most popular vision model, Convolutional Neural Networks (CNNs), has reached bottlenecks due to the intrinsic locality of convolution operations. Recently, transformers have been widely adopted for dense prediction for their capability to capture long-range visual dependence. However, due to the high computational complexity and large memory consumption of self-attention operations, transformers are usually used at downsampled feature resolutions. Such usage cannot effectively leverage the tissue-level textural information available only at the full image resolution. This textural information is crucial for medical dense prediction as it can differentiate the subtle human anatomy in medical images. In this study, we hypothesize that Multi-layer Perceptrons (MLPs) are superior alternatives to transformers in medical dense prediction where tissue-level details dominate the performance, as MLPs enable long-range dependence at the full image resolution. To validate our hypothesis, we develop a full-resolution hierarchical MLP framework that uses MLPs beginning from the full image resolution. We evaluate this framework with various MLP blocks on a wide range of medical dense prediction tasks including restoration, registration, and segmentation. Extensive experiments on six public well-benchmarked datasets show that, by simply using MLPs at full resolution, our framework outperforms its CNN and transformer counterparts and achieves state-of-the-art performance on various medical dense prediction tasks.