Scaling the Scattering Transform: Deep Hybrid Networks

TL;DR

This paper introduces a hybrid deep network approach that combines scattering transforms with learned layers, achieving competitive accuracy on ImageNet and improved performance in small sample scenarios by leveraging fixed representations and geometric priors.

Contribution

It demonstrates that fixed scattering features can replace learned initial layers in deep networks, enabling high accuracy with fewer layers and better performance in limited data regimes.

Findings

Achieves AlexNet-level accuracy on ImageNet using fixed scattering features.

Combines scattering transforms with ResNet to match ResNet-18 performance with fewer layers.

Outperforms end-to-end models in small sample settings by incorporating geometric priors.

Abstract

We use the scattering network as a generic and fixed ini-tialization of the first layers of a supervised hybrid deep network. We show that early layers do not necessarily need to be learned, providing the best results to-date with pre-defined representations while being competitive with Deep CNNs. Using a shallow cascade of 1 x 1 convolutions, which encodes scattering coefficients that correspond to spatial windows of very small sizes, permits to obtain AlexNet accuracy on the imagenet ILSVRC2012. We demonstrate that this local encoding explicitly learns invariance w.r.t. rotations. Combining scattering networks with a modern ResNet, we achieve a single-crop top 5 error of 11.4% on imagenet ILSVRC2012, comparable to the Resnet-18 architecture, while utilizing only 10 layers. We also find that hybrid architectures can yield excellent performance in the small sample regime, exceeding their end-to-end counterparts, through their ability to incorporate geometrical priors. We demonstrate this on subsets of the CIFAR-10 dataset and on the STL-10 dataset.