Enhancing Generalization in Convolutional Neural Networks through Regularization with Edge and Line Features

TL;DR

This paper introduces a regularization method for CNNs that biases the network toward using edge and line features, leading to improved generalization especially on small datasets by constraining convolution layers with fixed, pre-defined filters.

Contribution

The paper presents a novel regularization technique using fixed edge and line detection kernels in CNNs, enhancing generalization without increasing model complexity.

Findings

Test accuracy improved by 5-11 percentage points on four datasets.

Using nine or more pre-defined filters yields optimal performance.

Fixed filters have a low impact on recognition performance based on their dimensional span.

Abstract

This paper proposes a novel regularization approach to bias Convolutional Neural Networks (CNNs) toward utilizing edge and line features in their hidden layers. Rather than learning arbitrary kernels, we constrain the convolution layers to edge and line detection kernels. This intentional bias regularizes the models, improving generalization performance, especially on small datasets. As a result, test accuracies improve by margins of 5-11 percentage points across four challenging fine-grained classification datasets with limited training data and an identical number of trainable parameters. Instead of traditional convolutional layers, we use Pre-defined Filter Modules, which convolve input data using a fixed set of 3x3 pre-defined edge and line filters. A subsequent ReLU erases information that did not trigger any positive response. Next, a 1x1 convolutional layer generates linear combinations. Notably, the pre-defined filters are a fixed component of the architecture, remaining unchanged during the training phase. Our findings reveal that the number of dimensions spanned by the set of pre-defined filters has a low impact on recognition performance. However, the size of the set of filters matters, with nine or more filters providing optimal results.