Visualization Regularizers for Neural Network based Image Recognition

TL;DR

This paper introduces a visualization regularizer for neural networks in image recognition that enforces feature smoothness, improving accuracy without increasing training complexity.

Contribution

It proposes a novel visualization regularizer based on Tikhonov regularization that enhances feature smoothness and accuracy in neural networks for image tasks.

Findings

Higher classification accuracy than L2 and dropout regularizers

Enforces smoothness of hidden layer features

No additional computational cost during training

Abstract

The success of deep neural networks is mostly due their ability to learn meaningful features from the data. Features learned in the hidden layers of deep neural networks trained in computer vision tasks have been shown to be similar to mid-level vision features. We leverage this fact in this work and propose the visualization regularizer for image tasks. The proposed regularization technique enforces smoothness of the features learned by hidden nodes and turns out to be a special case of Tikhonov regularization. We achieve higher classification accuracy as compared to existing regularizers such as the L2 norm regularizer and dropout, on benchmark datasets without changing the training computational complexity.