Adaptive Hierarchical Decomposition of Large Deep Networks

TL;DR

This paper presents an adaptive hierarchical decomposition framework that automatically configures multiple smaller deep networks based on class similarities, improving scalability, training practicality, and classification accuracy for large datasets.

Contribution

It introduces a novel method to automatically analyze and organize deep networks hierarchically, optimizing their structure according to class similarities and problem complexity.

Findings

Hierarchical classifiers outperform single large classifiers in accuracy.

Smaller networks are more scalable and easier to train.

Adaptive configuration selection improves model performance.

Abstract

Deep learning has recently demonstrated its ability to rival the human brain for visual object recognition. As datasets get larger, a natural question to ask is if existing deep learning architectures can be extended to handle the 50+K classes thought to be perceptible by a typical human. Most deep learning architectures concentrate on splitting diverse categories, while ignoring the similarities amongst them. This paper introduces a framework that automatically analyzes and configures a family of smaller deep networks as a replacement to a singular, larger network. Class similarities guide the creation of a family from course to fine classifiers which solve categorical problems more effectively than a single large classifier. The resulting smaller networks are highly scalable, parallel and more practical to train, and achieve higher classification accuracy. This paper also proposes a method to adaptively select the configuration of the hierarchical family of classifiers using linkage statistics from overall and sub-classification confusion matrices. Depending on the number of classes and the complexity of the problem, a deep learning model is selected and the complexity is determined. Numerous experiments on network classes, layers, and architecture configurations validate our results.