Task Agnostic and Post-hoc Unseen Distribution Detection

TL;DR

This paper introduces TAPUDD, a task-agnostic, post-hoc clustering-based method for detecting unseen data distributions that leverages feature clustering and ensembling to improve detection across diverse tasks.

Contribution

The paper proposes TAPUDD, a novel clustering and ensembling approach that detects unseen distributions without task-specific tuning, applicable post-hoc to trained models.

Findings

Effective detection of unseen samples across multiple datasets

Outperforms or matches existing baselines in various tasks

Eliminates need for optimal cluster number tuning

Abstract

Despite the recent advances in out-of-distribution(OOD) detection, anomaly detection, and uncertainty estimation tasks, there do not exist a task-agnostic and post-hoc approach. To address this limitation, we design a novel clustering-based ensembling method, called Task Agnostic and Post-hoc Unseen Distribution Detection (TAPUDD) that utilizes the features extracted from the model trained on a specific task. Explicitly, it comprises of TAP-Mahalanobis, which clusters the training datasets' features and determines the minimum Mahalanobis distance of the test sample from all clusters. Further, we propose the Ensembling module that aggregates the computation of iterative TAP-Mahalanobis for a different number of clusters to provide reliable and efficient cluster computation. Through extensive experiments on synthetic and real-world datasets, we observe that our approach can detect unseen samples effectively across diverse tasks and performs better or on-par with the existing baselines. To this end, we eliminate the necessity of determining the optimal value of the number of clusters and demonstrate that our method is more viable for large-scale classification tasks.