A Simple Strategy to Provable Invariance via Orbit Mapping

TL;DR

This paper introduces a provably invariant neural network method using orbit mapping, which selects a canonical element from each transformation orbit to enhance robustness and efficiency.

Contribution

The paper presents a novel approach to achieve provable invariance in neural networks by orbit mapping, simplifying the design of invariant architectures.

Findings

Improves robustness to rotations and scalings in image and 3D point cloud data.

Demonstrates computational efficiency over traditional invariance methods.

Provides theoretical guarantees of invariance for specific transformations.

Abstract

Many applications require robustness, or ideally invariance, of neural networks to certain transformations of input data. Most commonly, this requirement is addressed by training data augmentation, using adversarial training, or defining network architectures that include the desired invariance by design. In this work, we propose a method to make network architectures provably invariant with respect to group actions by choosing one element from a (possibly continuous) orbit based on a fixed criterion. In a nutshell, we intend to 'undo' any possible transformation before feeding the data into the actual network. Further, we empirically analyze the properties of different approaches which incorporate invariance via training or architecture, and demonstrate the advantages of our method in terms of robustness and computational efficiency. In particular, we investigate the robustness with respect to rotations of images (which can hold up to discretization artifacts) as well as the provable orientation and scaling invariance of 3D point cloud classification.