Stochastic Pooling for Regularization of Deep Convolutional Neural Networks

TL;DR

This paper proposes stochastic pooling, a regularization technique for deep convolutional neural networks that replaces deterministic pooling with a probabilistic approach, improving performance without extra hyperparameters.

Contribution

It introduces a hyper-parameter free stochastic pooling method that enhances regularization and can be combined with existing techniques, achieving state-of-the-art results.

Findings

Achieves state-of-the-art performance on four image datasets.

Compatible with other regularization methods like dropout.

Effective in improving generalization of deep CNNs.

Abstract

We introduce a simple and effective method for regularizing large convolutional neural networks. We replace the conventional deterministic pooling operations with a stochastic procedure, randomly picking the activation within each pooling region according to a multinomial distribution, given by the activities within the pooling region. The approach is hyper-parameter free and can be combined with other regularization approaches, such as dropout and data augmentation. We achieve state-of-the-art performance on four image datasets, relative to other approaches that do not utilize data augmentation.