Deformation Robust Roto-Scale-Translation Equivariant CNNs

TL;DR

This paper introduces a new RST-CNN model that achieves joint rotation, scaling, and translation equivariance, along with deformation robustness, leading to improved generalization especially with limited data.

Contribution

The paper proposes the first RST-CNN that guarantees equivariance over rotation, scaling, and translation simultaneously, and provides a stability analysis for deformation robustness.

Findings

Significant accuracy improvements on MNIST, Fashion-MNIST, STL-10

Enhanced out-of-distribution generalization due to deformation robustness

Effective small data regime performance with multiple transformations

Abstract

Incorporating group symmetry directly into the learning process has proved to be an effective guideline for model design. By producing features that are guaranteed to transform covariantly to the group actions on the inputs, group-equivariant convolutional neural networks (G-CNNs) achieve significantly improved generalization performance in learning tasks with intrinsic symmetry. General theory and practical implementation of G-CNNs have been studied for planar images under either rotation or scaling transformation, but only individually. We present, in this paper, a roto-scale-translation equivariant CNN (RST-CNN), that is guaranteed to achieve equivariance jointly over these three groups via coupled group convolutions. Moreover, as symmetry transformations in reality are rarely perfect and typically subject to input deformation, we provide a stability analysis of the equivariance of representation to input distortion, which motivates the truncated expansion of the convolutional filters under (pre-fixed) low-frequency spatial modes. The resulting model provably achieves deformation-robust RST equivariance, i.e., the RST symmetry is still "approximately" preserved when the transformation is "contaminated" by a nuisance data deformation, a property that is especially important for out-of-distribution generalization. Numerical experiments on MNIST, Fashion-MNIST, and STL-10 demonstrate that the proposed model yields remarkable gains over prior arts, especially in the small data regime where both rotation and scaling variations are present within the data.