Deep is a Luxury We Don't Have

TL;DR

This paper introduces HCT, a high-resolution vision transformer that efficiently models long-range dependencies in medical images using linear self-attention, outperforming CNNs on mammography data.

Contribution

We propose HCT, a novel high-resolution transformer model utilizing linear self-attention to reduce complexity and improve performance on medical imaging tasks.

Findings

HCT outperforms CNN counterparts on mammography datasets.

HCT effectively models long-range dependencies in high-resolution images.

HCT demonstrates a suitable receptive field for medical image analysis.

Abstract

Medical images come in high resolutions. A high resolution is vital for finding malignant tissues at an early stage. Yet, this resolution presents a challenge in terms of modeling long range dependencies. Shallow transformers eliminate this problem, but they suffer from quadratic complexity. In this paper, we tackle this complexity by leveraging a linear self-attention approximation. Through this approximation, we propose an efficient vision model called HCT that stands for High resolution Convolutional Transformer. HCT brings transformers' merits to high resolution images at a significantly lower cost. We evaluate HCT using a high resolution mammography dataset. HCT is significantly superior to its CNN counterpart. Furthermore, we demonstrate HCT's fitness for medical images by evaluating its effective receptive field.Code available at https://bit.ly/3ykBhhf