Tree-CNN: A Hierarchical Deep Convolutional Neural Network for Incremental Learning

TL;DR

This paper introduces Tree-CNN, a hierarchical deep learning model that incrementally learns new classes by growing in a tree structure, reducing retraining effort and maintaining accuracy in evolving data environments.

Contribution

The paper presents a novel self-growing hierarchical CNN architecture that effectively handles incremental learning and class expansion without catastrophic forgetting.

Findings

Reduces training effort compared to fine-tuning.

Maintains competitive accuracy on CIFAR-10 and CIFAR-100.

Supports incremental addition of new classes in a hierarchical manner.

Abstract

Over the past decade, Deep Convolutional Neural Networks (DCNNs) have shown remarkable performance in most computer vision tasks. These tasks traditionally use a fixed dataset, and the model, once trained, is deployed as is. Adding new information to such a model presents a challenge due to complex training issues, such as "catastrophic forgetting", and sensitivity to hyper-parameter tuning. However, in this modern world, data is constantly evolving, and our deep learning models are required to adapt to these changes. In this paper, we propose an adaptive hierarchical network structure composed of DCNNs that can grow and learn as new data becomes available. The network grows in a tree-like fashion to accommodate new classes of data, while preserving the ability to distinguish the previously trained classes. The network organizes the incrementally available data into feature-driven super-classes and improves upon existing hierarchical CNN models by adding the capability of self-growth. The proposed hierarchical model, when compared against fine-tuning a deep network, achieves significant reduction of training effort, while maintaining competitive accuracy on CIFAR-10 and CIFAR-100.