A Bayesian Approach to Invariant Deep Neural Networks

TL;DR

This paper introduces a Bayesian neural network that learns invariances directly from data by inferring weight-sharing schemes, outperforming non-invariant models even without data augmentation.

Contribution

It presents a novel Bayesian architecture capable of learning invariances from data, advancing invariant neural network design.

Findings

Outperforms non-invariant architectures on invariant datasets

Effective without data augmentation

Learns invariances directly from data

Abstract

We propose a novel Bayesian neural network architecture that can learn invariances from data alone by inferring a posterior distribution over different weight-sharing schemes. We show that our model outperforms other non-invariant architectures, when trained on datasets that contain specific invariances. The same holds true when no data augmentation is performed.