Scale Steerable Filters for Locally Scale-Invariant Convolutional Neural Networks

TL;DR

This paper introduces a scale-steerable filter basis for locally scale-invariant CNNs, improving performance and structure preservation on scaled datasets and matching the generalization of affine transformation methods.

Contribution

The paper proposes a novel scale-steerable filter basis called log-radial harmonics for locally scale-invariant CNNs, enhancing performance and interpretability.

Findings

Significant performance improvements on MNIST-Scale and FMNIST-Scale datasets.

Filters exhibit meaningful structure and higher spatial-structure preservation.

Comparable generalization to affine transformation methods like Spatial Transformers.

Abstract

Augmenting transformation knowledge onto a convolutional neural network's weights has often yielded significant improvements in performance. For rotational transformation augmentation, an important element to recent approaches has been the use of a steerable basis i.e. the circular harmonics. Here, we propose a scale-steerable filter basis for the locally scale-invariant CNN, denoted as log-radial harmonics. By replacing the kernels in the locally scale-invariant CNN \cite{lsi_cnn} with scale-steered kernels, significant improvements in performance can be observed on the MNIST-Scale and FMNIST-Scale datasets. Training with a scale-steerable basis results in filters which show meaningful structure, and feature maps demonstrate which demonstrate visibly higher spatial-structure preservation of input. Furthermore, the proposed scale-steerable CNN shows on-par generalization to global affine transformation estimation methods such as Spatial Transformers, in response to test-time data distortions.