Autoselection of the Ensemble of Convolutional Neural Networks with Second-Order Cone Programming

TL;DR

This paper introduces a mathematical approach using second-order cone programming to optimally prune CNN ensembles, enhancing accuracy and diversity while reducing computational complexity on standard datasets.

Contribution

It presents a novel sparse second-order cone optimization model for ensemble pruning of CNNs, balancing accuracy, diversity, and complexity reduction.

Findings

Effective ensemble pruning on CIFAR-10, CIFAR-100, and MNIST datasets.

Significant reduction in model complexity with maintained or improved accuracy.

Promising results demonstrating the method's potential in deep learning applications.

Abstract

Ensemble techniques are frequently encountered in machine learning and engineering problems since the method combines different models and produces an optimal predictive solution. The ensemble concept can be adapted to deep learning models to provide robustness and reliability. Due to the growth of the models in deep learning, using ensemble pruning is highly important to deal with computational complexity. Hence, this study proposes a mathematical model which prunes the ensemble of Convolutional Neural Networks (CNN) consisting of different depths and layers that maximizes accuracy and diversity simultaneously with a sparse second order conic optimization model. The proposed model is tested on CIFAR-10, CIFAR-100 and MNIST data sets which gives promising results while reducing the complexity of models, significantly.