Regularization and Optimization strategies in Deep Convolutional Neural Network

TL;DR

This paper reviews the theoretical concepts and mathematical formulations of regularization and optimization strategies used to improve deep convolutional neural networks, addressing issues like overfitting, computational cost, and convergence.

Contribution

It provides a comprehensive overview of the theoretical foundations and mathematical details of key regularization and optimization techniques in ConvNets.

Findings

Regularization techniques improve generalization in ConvNets.

Optimization strategies enhance training efficiency and convergence.

These methods significantly boost network performance and reduce computational costs.

Abstract

Convolution Neural Networks, known as ConvNets exceptionally perform well in many complex machine learning tasks. The architecture of ConvNets demands the huge and rich amount of data and involves with a vast number of parameters that leads the learning takes to be computationally expensive, slow convergence towards the global minima, trap in local minima with poor predictions. In some cases, architecture overfits the data and make the architecture difficult to generalise for new samples that were not in the training set samples. To address these limitations, many regularization and optimization strategies are developed for the past few years. Also, studies suggested that these techniques significantly increase the performance of the networks as well as reducing the computational cost. In implementing these techniques, one must thoroughly understand the theoretical concept of how this technique works in increasing the expressive power of the networks. This article is intended to provide the theoretical concepts and mathematical formulation of the most commonly used strategies in developing a ConvNet architecture.