Streaming Networks: Increase Noise Robustness and Filter Diversity via Hard-wired and Input-induced Sparsity

TL;DR

This paper introduces Streaming Networks, a novel architecture that enhances noise robustness and filter diversity in CNNs by leveraging hard-wired and input-induced sparsity, demonstrating superior performance on noisy images.

Contribution

The paper proposes Streaming Networks with combined sparsity techniques, showing improved noise robustness and increased filter diversity over traditional CNNs.

Findings

Streaming Networks outperform traditional CNNs in noisy image recognition.

Presence of both hard-wired and input-induced sparsity is crucial for robustness.

Filter weight distribution becomes more uniform with increased sparsity.

Abstract

The CNNs have achieved a state-of-the-art performance in many applications. Recent studies illustrate that CNN's recognition accuracy drops drastically if images are noise corrupted. We focus on the problem of robust recognition accuracy of noise-corrupted images. We introduce a novel network architecture called Streaming Networks. Each stream is taking a certain intensity slice of the original image as an input, and stream parameters are trained independently. We use network capacity, hard-wired and input-induced sparsity as the dimensions for experiments. The results indicate that only the presence of both hard-wired and input-induces sparsity enables robust noisy image recognition. Streaming Nets is the only architecture which has both types of sparsity and exhibits higher robustness to noise. Finally, to illustrate increase in filter diversity we illustrate that a distribution of filter weights of the first conv layer gradually approaches uniform distribution as the degree of hard-wired and domain-induced sparsity and capacities increases.