FLatS: Principled Out-of-Distribution Detection with Feature-Based Likelihood Ratio Score

TL;DR

This paper introduces FLatS, a likelihood ratio-based framework for out-of-distribution detection in NLP, which outperforms existing methods by theoretically addressing their limitations and serving as a versatile enhancement tool.

Contribution

The paper proposes FLatS, a principled likelihood ratio approach for OOD detection, improving over existing feature-based methods and providing a general framework for enhancement.

Findings

FLatS achieves new state-of-the-art results on benchmarks.

Likelihood ratio approach outperforms density estimation methods.

Framework can enhance other OOD detection techniques.

Abstract

Detecting out-of-distribution (OOD) instances is crucial for NLP models in practical applications. Although numerous OOD detection methods exist, most of them are empirical. Backed by theoretical analysis, this paper advocates for the measurement of the "OOD-ness" of a test case $\boldsymbol{x}$ through the likelihood ratio between out-distribution $\mathcal P_{\textit{out}}$ and in-distribution $\mathcal P_{\textit{in}}$. We argue that the state-of-the-art (SOTA) feature-based OOD detection methods, such as Maha and KNN, are suboptimal since they only estimate in-distribution density $p_{\textit{in}}(\boldsymbol{x})$. To address this issue, we propose FLatS, a principled solution for OOD detection based on likelihood ratio. Moreover, we demonstrate that FLatS can serve as a general framework capable of enhancing other OOD detection methods by incorporating out-distribution density $p_{\textit{out}}(\boldsymbol{x})$ estimation. Experiments show that FLatS establishes a new SOTA on popular benchmarks. Our code is publicly available at https://github.com/linhaowei1/FLatS.