Tree-based Ensemble Learning for Out-of-distribution Detection

TL;DR

This paper introduces TOOD detection, a tree-based method for out-of-distribution detection that uses pairwise Hamming distances of tree embeddings to effectively distinguish in-distribution from out-of-distribution samples across multiple data types.

Contribution

The paper presents a novel, interpretable, and efficient tree-based out-of-distribution detection method that generalizes well to various data modalities and unsupervised settings.

Findings

Outperforms state-of-the-art OOD detection methods on tabular, image, and text data.

Provides a robust and interpretable approach based on tree embeddings.

Efficient and flexible for different machine learning tasks.

Abstract

Being able to successfully determine whether the testing samples has similar distribution as the training samples is a fundamental question to address before we can safely deploy most of the machine learning models into practice. In this paper, we propose TOOD detection, a simple yet effective tree-based out-of-distribution (TOOD) detection mechanism to determine if a set of unseen samples will have similar distribution as of the training samples. The TOOD detection mechanism is based on computing pairwise hamming distance of testing samples' tree embeddings, which are obtained by fitting a tree-based ensemble model through in-distribution training samples. Our approach is interpretable and robust for its tree-based nature. Furthermore, our approach is efficient, flexible to various machine learning tasks, and can be easily generalized to unsupervised setting. Extensive experiments are conducted to show the proposed method outperforms other state-of-the-art out-of-distribution detection methods in distinguishing the in-distribution from out-of-distribution on various tabular, image, and text data.