Learning unfolded networks with a cyclic group structure

TL;DR

This paper introduces interpretable neural networks that explicitly encode rotation equivariance using unfolded architectures, achieving competitive performance with fewer parameters on rotated image datasets.

Contribution

It presents a novel method combining unfolded networks with equivariance to domain knowledge, enhancing interpretability and efficiency.

Findings

Competitive accuracy on rotated MNIST and CIFAR-10

Fewer parameters than baseline models

Interpretable sparse activations

Abstract

Deep neural networks lack straightforward ways to incorporate domain knowledge and are notoriously considered black boxes. Prior works attempted to inject domain knowledge into architectures implicitly through data augmentation. Building on recent advances on equivariant neural networks, we propose networks that explicitly encode domain knowledge, specifically equivariance with respect to rotations. By using unfolded architectures, a rich framework that originated from sparse coding and has theoretical guarantees, we present interpretable networks with sparse activations. The equivariant unfolded networks compete favorably with baselines, with only a fraction of their parameters, as showcased on (rotated) MNIST and CIFAR-10.