Scale-covariant and scale-invariant Gaussian derivative networks

TL;DR

This paper introduces a hybrid deep learning architecture based on scale-space theory that is provably scale covariant and invariant, enabling robust image classification across multiple scales.

Contribution

It combines scale-space primitives with deep learning to create a network that is both scale covariant and invariant, with proven theoretical properties.

Findings

Achieves scale generalization on MNISTLargeScale dataset

Demonstrates robustness to rescaled images not seen during training

Provides a provably scale-invariant and covariant neural network architecture

Abstract

This paper presents a hybrid approach between scale-space theory and deep learning, where a deep learning architecture is constructed by coupling parameterized scale-space operations in cascade. By sharing the learnt parameters between multiple scale channels, and by using the transformation properties of the scale-space primitives under scaling transformations, the resulting network becomes provably scale covariant. By in addition performing max pooling over the multiple scale channels, a resulting network architecture for image classification also becomes provably scale invariant. We investigate the performance of such networks on the MNISTLargeScale dataset, which contains rescaled images from original MNIST over a factor of 4 concerning training data and over a factor of 16 concerning testing data. It is demonstrated that the resulting approach allows for scale generalization, enabling good performance for classifying patterns at scales not present in the training data.