Imaging with Equivariant Deep Learning

TL;DR

This paper reviews how incorporating symmetry and equivariance into deep learning models enhances computational imaging, addressing challenges posed by noisy, non-equivariant forward operators and improving generalization and imaging capabilities.

Contribution

It introduces the emerging field of equivariant imaging, highlighting how symmetry principles can be integrated into imaging models to overcome unique challenges.

Findings

Equivariant imaging improves generalization in computational imaging.

Symmetry principles can be linked to acquisition physics and iterative reconstruction.

The approach offers new opportunities for imaging with noisy, ill-conditioned data.

Abstract

From early image processing to modern computational imaging, successful models and algorithms have relied on a fundamental property of natural signals: symmetry. Here symmetry refers to the invariance property of signal sets to transformations such as translation, rotation or scaling. Symmetry can also be incorporated into deep neural networks in the form of equivariance, allowing for more data-efficient learning. While there has been important advances in the design of end-to-end equivariant networks for image classification in recent years, computational imaging introduces unique challenges for equivariant network solutions since we typically only observe the image through some noisy ill-conditioned forward operator that itself may not be equivariant. We review the emerging field of equivariant imaging and show how it can provide improved generalization and new imaging opportunities. Along the way we show the interplay between the acquisition physics and group actions and links to iterative reconstruction, blind compressed sensing and self-supervised learning.