Truly shift-equivariant convolutional neural networks with adaptive polyphase upsampling

TL;DR

This paper introduces adaptive polyphase upsampling (APS-U) to achieve perfect shift equivariance in CNNs used for image reconstruction, improving robustness and performance without sacrificing image quality.

Contribution

The paper proposes a novel non-linear upsampling method, APS-U, enabling CNNs with encoder-decoder architectures to be perfectly shift equivariant, extending previous APS-D methods.

Findings

APS-D/U layers achieve state-of-the-art shift equivariance in MRI and CT reconstruction.

APS-D/U improves robustness of CNNs to shifts beyond training distribution.

The method maintains high image reconstruction quality.

Abstract

Convolutional neural networks lack shift equivariance due to the presence of downsampling layers. In image classification, adaptive polyphase downsampling (APS-D) was recently proposed to make CNNs perfectly shift invariant. However, in networks used for image reconstruction tasks, it can not by itself restore shift equivariance. We address this problem by proposing adaptive polyphase upsampling (APS-U), a non-linear extension of conventional upsampling, which allows CNNs with symmetric encoder-decoder architecture (for example U-Net) to exhibit perfect shift equivariance. With MRI and CT reconstruction experiments, we show that networks containing APS-D/U layers exhibit state of the art equivariance performance without sacrificing on image reconstruction quality. In addition, unlike prior methods like data augmentation and anti-aliasing, the gains in equivariance obtained from APS-D/U also extend to images outside the training distribution.