On Binary Classification with Single-Layer Convolutional Neural Networks

TL;DR

This paper investigates the design factors of single-layer convolutional neural networks for object classification, emphasizing the importance of pre-training, regularization, and pool size, and demonstrates competitive performance on cats and dogs classification.

Contribution

It provides insights into effective design choices for simple single-layer CNNs, highlighting the roles of pre-training and regularization, and compares performance with more complex models.

Findings

Pre-training with unsupervised schemes is crucial.

Strong regularizers like dropout can be harmful.

Performance on cats and dogs classification approaches state-of-the-art.

Abstract

Convolutional neural networks are becoming standard tools for solving object recognition and visual tasks. However, most of the design and implementation of these complex models are based on trail-and-error. In this report, the main focus is to consider some of the important factors in designing convolutional networks to perform better. Specifically, classification with wide single-layer networks with large kernels as a general framework is considered. Particularly, we will show that pre-training using unsupervised schemes is vital, reasonable regularization is beneficial and applying of strong regularizers like dropout could be devastating. Pool size is also could be as important as learning procedure itself. In addition, it has been presented that using such a simple and relatively fast model for classifying cats and dogs, performance is close to state-of-the-art achievable by a combination of SVM models on color and texture features.