On the Learning with Augmented Class via Forests

TL;DR

This paper introduces LACForest, a novel approach that incorporates augmented classes into decision forests using a new impurity measure, improving classification when testing data includes unseen classes.

Contribution

It proposes augmented Gini impurity for decision trees, enabling learning with unseen classes, and develops neural forests leveraging this impurity for enhanced performance.

Findings

LACForest outperforms traditional methods on augmented class tasks.

The augmented Gini impurity converges theoretically.

Neural forests with augmented Gini impurity improve representation learning.

Abstract

Decision trees and forests have achieved successes in various real applications, most working with all testing classes known in training data. In this work, we focus on learning with augmented class via forests, where an augmented class may appear in testing data yet not in training data. We incorporate information of augmented class into trees' splitting, that is, augmented Gini impurity, a new splitting criterion is introduced to exploit some unlabeled data from testing distribution. We then develop the Learning with Augmented Class via Forests (short for LACForest) approach, which constructs shallow forests according to the augmented Gini impurity and then splits forests with pseudo-labeled augmented instances for better performance. We also develop deep neural forests via an optimization objective based on our augmented Gini impurity, which essentially utilizes the representation power of neural networks for forests. Theoretically, we present the convergence analysis for our augmented Gini impurity, and we finally conduct experiments to evaluate our approaches. The code is available at https://github.com/nju-xuf/LACForest.