Adaptive Decision Forest: An Incremental Machine Learning Framework

TL;DR

This paper introduces Adaptive Decision Forest (ADF), an incremental learning framework capable of classifying new data, handling unseen classes and concept drift, and efficiently managing big data, demonstrating superior performance over existing methods.

Contribution

The paper proposes a novel incremental decision forest framework with the iSAT splitting strategy, enabling classification of unseen classes and handling concept drift without forgetting prior knowledge.

Findings

ADF outperforms eight state-of-the-art techniques on multiple datasets.

The iSAT splitting strategy effectively handles unseen classes.

ADF demonstrates robustness to concept drift and scalability to big data.

Abstract

In this study, we present an incremental machine learning framework called Adaptive Decision Forest (ADF), which produces a decision forest to classify new records. Based on our two novel theorems, we introduce a new splitting strategy called iSAT, which allows ADF to classify new records even if they are associated with previously unseen classes. ADF is capable of identifying and handling concept drift; it, however, does not forget previously gained knowledge. Moreover, ADF is capable of handling big data if the data can be divided into batches. We evaluate ADF on five publicly available natural data sets and one synthetic data set, and compare the performance of ADF against the performance of eight state-of-the-art techniques. Our experimental results, including statistical sign test and Nemenyi test analyses, indicate a clear superiority of the proposed framework over the state-of-the-art techniques.