Analytic Learning of Convolutional Neural Network For Pattern Recognition

TL;DR

This paper introduces an analytic learning method for CNNs that trains in one epoch, offering faster training with comparable accuracy to back-propagation, especially effective with limited data.

Contribution

It proposes ACnnL, a novel analytic CNN learning approach with a closed-form solution, explaining CNN generalization and improving training efficiency.

Findings

ACnnL trains CNNs significantly faster than BP.

ACnnL achieves comparable prediction accuracy to BP.

ACnnL performs well with scarce training data.

Abstract

Training convolutional neural networks (CNNs) with back-propagation (BP) is time-consuming and resource-intensive particularly in view of the need to visit the dataset multiple times. In contrast, analytic learning attempts to obtain the weights in one epoch. However, existing attempts to analytic learning considered only the multilayer perceptron (MLP). In this article, we propose an analytic convolutional neural network learning (ACnnL). Theoretically we show that ACnnL builds a closed-form solution similar to its MLP counterpart, but differs in their regularization constraints. Consequently, we are able to answer to a certain extent why CNNs usually generalize better than MLPs from the implicit regularization point of view. The ACnnL is validated by conducting classification tasks on several benchmark datasets. It is encouraging that the ACnnL trains CNNs in a significantly fast manner with reasonably close prediction accuracies to those using BP. Moreover, our experiments disclose a unique advantage of ACnnL under the small-sample scenario when training data are scarce or expensive.