MaxDropoutV2: An Improved Method to Drop out Neurons in Convolutional Neural Networks

TL;DR

MaxDropoutV2 is an enhanced regularization method for convolutional neural networks that improves training speed and accuracy over the original MaxDropout, addressing overfitting in complex models.

Contribution

This paper introduces MaxDropoutV2, an improved version of MaxDropout, offering faster training and better accuracy in CNNs for regularization.

Findings

MaxDropoutV2 trains faster than MaxDropout.

MaxDropoutV2 achieves higher accuracy in most cases.

MaxDropoutV2 effectively reduces overfitting.

Abstract

In the last decade, exponential data growth supplied the machine learning-based algorithms' capacity and enabled their usage in daily life activities. Additionally, such an improvement is partially explained due to the advent of deep learning techniques, i.e., stacks of simple architectures that end up in more complex models. Although both factors produce outstanding results, they also pose drawbacks regarding the learning process since training complex models denotes an expensive task and results are prone to overfit the training data. A supervised regularization technique called MaxDropout was recently proposed to tackle the latter, providing several improvements concerning traditional regularization approaches. In this paper, we present its improved version called MaxDropoutV2. Results considering two public datasets show that the model performs faster than the standard version and, in most cases, provides more accurate results.