FocusedDropout for Convolutional Neural Network

TL;DR

This paper introduces FocusedDropout, a non-random dropout method for CNNs that enhances target focus and improves classification performance across multiple datasets by selectively retaining target-related features.

Contribution

The paper proposes a novel non-random dropout technique called FocusedDropout that searches for and retains target-related features to improve CNN performance.

Findings

Improves accuracy on CIFAR10, CIFAR100, Tiny ImageNet

Enhances target focus in CNNs

Effective with only 10% batch usage

Abstract

In convolutional neural network (CNN), dropout cannot work well because dropped information is not entirely obscured in convolutional layers where features are correlated spatially. Except randomly discarding regions or channels, many approaches try to overcome this defect by dropping influential units. In this paper, we propose a non-random dropout method named FocusedDropout, aiming to make the network focus more on the target. In FocusedDropout, we use a simple but effective way to search for the target-related features, retain these features and discard others, which is contrary to the existing methods. We found that this novel method can improve network performance by making the network more target-focused. Besides, increasing the weight decay while using FocusedDropout can avoid the overfitting and increase accuracy. Experimental results show that even a slight cost, 10\% of batches employing FocusedDropout, can produce a nice performance boost over the baselines on multiple datasets of classification, including CIFAR10, CIFAR100, Tiny Imagenet, and has a good versatility for different CNN models.