Ensembles of feedforward-designed convolutional neural networks

TL;DR

This paper proposes an ensemble approach for feedforward-designed CNNs that enhances image classification accuracy by increasing model diversity through various strategies and tailoring ensembles for hard samples, validated on MNIST and CIFAR-10.

Contribution

It introduces a novel ensemble method for FF-CNNs that improves performance by diversifying models and focusing on hard samples, a new approach in this domain.

Findings

Ensemble method improves classification accuracy on MNIST and CIFAR-10.

Diversity strategies significantly enhance ensemble performance.

Tailored ensemble for hard samples boosts accuracy further.

Abstract

An ensemble method that fuses the output decision vectors of multiple feedforward-designed convolutional neural networks (FF-CNNs) to solve the image classification problem is proposed in this work. To enhance the performance of the ensemble system, it is critical to increasing the diversity of FF-CNN models. To achieve this objective, we introduce diversities by adopting three strategies: 1) different parameter settings in convolutional layers, 2) flexible feature subsets fed into the Fully-connected (FC) layers, and 3) multiple image embeddings of the same input source. Furthermore, we partition input samples into easy and hard ones based on their decision confidence scores. As a result, we can develop a new ensemble system tailored to hard samples to further boost classification accuracy. Experiments are conducted on the MNIST and CIFAR-10 datasets to demonstrate the effectiveness of the ensemble method.