Combining Deep Learning with Good Old-Fashioned Machine Learning

TL;DR

This paper introduces Deep GOld, a stacking-based framework that combines pretrained deep networks with traditional machine learning algorithms, resulting in consistent performance improvements on multiple image classification datasets.

Contribution

The paper presents a novel ensemble framework that effectively integrates deep learning with classical machine learning, demonstrating significant performance gains.

Findings

Deep GOld improves performance in 110 out of 120 experiments.

Ensemble selection from 51 deep networks enhances accuracy.

Combining deep learning with traditional ML yields consistent benefits.

Abstract

We present a comprehensive, stacking-based framework for combining deep learning with good old-fashioned machine learning, called Deep GOld. Our framework involves ensemble selection from 51 retrained pretrained deep networks as first-level models, and 10 machine-learning algorithms as second-level models. Enabled by today's state-of-the-art software tools and hardware platforms, Deep GOld delivers consistent improvement when tested on four image-classification datasets: Fashion MNIST, CIFAR10, CIFAR100, and Tiny ImageNet. Of 120 experiments, in all but 10 Deep GOld improved the original networks' performance.